Re: [Zope] zeo clients not synchronized, modifications not shown
Hi Christopher, Great pointers, thank you very much for your response! I use pound for load balancing (from the beginning - nothing changed here). I will investigate using your indications. Thanks again, Dragos On Thu, Nov 5, 2015 at 2:03 PM, Christopher Lozinski < lozin...@freerecruiting.com> wrote: > > > On 11/5/15 12:29 PM, Dragos Chirila wrote: > > My impression is that the save itself is handled by lets say client1 > > and then the redirect and render of the edit page by a different one - > > client2. > I don't know what is going on, but let me ask a few obvious questions. > How do you assign web requests to zeo clients. If the redirect is > handled by a different zeo client, then maybe it has not gotten the > cache invalidation. Maybe in your web page display, you can display the > process id, to verify if indeed it is a different zeo client which is > the source of the problem. At that point, put some logging into the > cache invalidations to see if they are happening > correctly, and before the new page is redisplayed. Of do something on > your load balancer to make sure that each ip is served by the same zeo > client. > > And why did the problem suddenly start happening? Wierd. > > Hope that helps. > -- Dragos Chirila objectval...@gmail.com (+4) 0722 395375 ___ Zope maillist - Zope@zope.org https://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] ZEO manageUndo not working - DisconnectedError
Good morning all,
after further testing we noticed our Zope2 virtualenv setup was done with the
ZMS3 (CMS) install routine using pip install ZMS3 --process-dependency-links.
This installed different package versions.
The original pip Zope2 uses these versions: http://pastebin.com/ZEwhVba0
The ZMS3 people use these versions: http://pastebin.com/iu43Ss5w
For example, the ZMS3 setup uses:
• ZopeUndo==4.0
• ZODB==4.1.0
• ZODB3==3.11.0
as the original Zope2 uses:
• ZODB3==3.10.5
No ZEO, no ZODB packages.
Why is Zope2 using the old 3.10 ZODB version and why is there no ZODB4
installed? ZEO server and clients still work fine without these packages.
The thing is, I notice a lot worse performance with the ZODB, especially with
traversal, using the native Zope2 setup routine. Catalog search takes almost
2-3 seconds compared to the ZMS3 versions with a local non-ZEO-Zope instance.
With the ZMS3 versions ZODB usage is a LOT faster, but there are problems with
Undo for example in combination with ZEO.
Any ideas on this? I don’t get this version jungle...
Best
Sebastian
Am 06.05.2015 um 07:42 schrieb Niels Dettenbach (Syndicat IT Internet)
n...@syndicat.com:
Am 6. Mai 2015 01:24:50 MESZ, schrieb Sebastian Tänzer s...@taenzer.me:
2015-05-06 01:16:46 WARNING ZEO.zrpc (19205) CW: error connecting to
('::1', ): ECONNREFUSED
2015-05-06 01:16:46 INFO ZEO.ClientStorage zeostorage Testing
connection ManagedClientConnection ('127.0.0.1', )
2015-05-06 01:16:46 INFO ZEO.zrpc.Connection('C') (127.0.0.1:)
hmm,
just a small shot in the dark: Even if this should work too - why the
client tries to connect the IPv4 address of loopback and the error gives the
IPv6 of it back?
I would try to eleminate this if possible and bring Zope down to IPv4
completely, because there are still many software libraries and applications
around (even some underlying os parts) which are not well tested with IPv6.
If this solves your prob - no idea...
hth a bit.
cheerioh,
Niels.
--
Niels Dettenbach
Syndicat IT Internet
http://www.syndicat.com
___
Zope maillist - Zope@zope.org
https://mail.zope.org/mailman/listinfo/zope
** No cross posts or HTML encoding! **
(Related lists -
https://mail.zope.org/mailman/listinfo/zope-announce
https://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] ZEO manageUndo not working - DisconnectedError
thanks for the „shot“, yet disabling IPv6 locally on the VM does not help, same
error:
(server):
2015-05-06T10:05:51 INFO ZEO.StorageServer (127.0.0.1:44833) disconnected
--
2015-05-06T10:05:51 INFO ZEO.zrpc.Connection('S') (127.0.0.1:44835) received
handshake 'Z3101'
--
2015-05-06T10:06:35 ERROR ZEO.zrpc (3040) can't decode message:
'(K\x05I00\nU\x08undoInfo(K\x00K\x14}U\x0bdescriptioncco...'
--
2015-05-06T10:06:35 ERROR ZEO.zrpc.Connection('S') (127.0.0.1:44835) Error
caught in asyncore
Traceback (most recent call last):
File /opt/python/2.7.9/lib/python2.7/asyncore.py, line 83, in read
obj.handle_read_event()
File /opt/python/2.7.9/lib/python2.7/asyncore.py, line 449, in
handle_read_event
self.handle_read()
File /opt/python/zope2-latest/lib/python2.7/site-packages/ZEO/zrpc/smac.py,
line 240, in handle_read
self.message_input(msg)
File
/opt/python/zope2-latest/lib/python2.7/site-packages/ZEO/zrpc/connection.py,
line 417, in message_input
msgid, async, name, args = self.decode(message)
File
/opt/python/zope2-latest/lib/python2.7/site-packages/ZEO/zrpc/marshal.py,
line 80, in server_decode
return unpickler.load() # msgid, flags, name, args
File
/opt/python/zope2-latest/lib/python2.7/site-packages/ZEO/zrpc/marshal.py,
line 120, in server_find_global
raise ZRPCError(import error %s: %s % (module, msg))
ZRPCError: import error copy_reg:
--
2015-05-06T10:06:35 INFO ZEO.StorageServer (127.0.0.1:44835) disconnected
--
2015-05-06T10:06:35 INFO ZEO.zrpc.Connection('S') (127.0.0.1:44837) received
handshake ‚Z3101'
More ideas? :-)
Am 06.05.2015 um 07:42 schrieb Niels Dettenbach (Syndicat IT Internet)
n...@syndicat.com:
Am 6. Mai 2015 01:24:50 MESZ, schrieb Sebastian Tänzer s...@taenzer.me:
2015-05-06 01:16:46 WARNING ZEO.zrpc (19205) CW: error connecting to
('::1', ): ECONNREFUSED
2015-05-06 01:16:46 INFO ZEO.ClientStorage zeostorage Testing
connection ManagedClientConnection ('127.0.0.1', )
2015-05-06 01:16:46 INFO ZEO.zrpc.Connection('C') (127.0.0.1:)
hmm,
just a small shot in the dark: Even if this should work too - why the
client tries to connect the IPv4 address of loopback and the error gives the
IPv6 of it back?
I would try to eleminate this if possible and bring Zope down to IPv4
completely, because there are still many software libraries and applications
around (even some underlying os parts) which are not well tested with IPv6.
If this solves your prob - no idea...
hth a bit.
cheerioh,
Niels.
--
Niels Dettenbach
Syndicat IT Internet
http://www.syndicat.com
___
Zope maillist - Zope@zope.org
https://mail.zope.org/mailman/listinfo/zope
** No cross posts or HTML encoding! **
(Related lists -
https://mail.zope.org/mailman/listinfo/zope-announce
https://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] ZEO manageUndo not working - DisconnectedError
After more „research“ I see that the ZMS3 pip packages uses completely
different versions that don’t seem to be compatible with ZEO (at least Undo is
not working). I’m aware „undo“ is not a really used feature at all as revisions
most of the time are beeing implemented by products. My only fear is that we’re
running into other problems with the ZODB if the base setup is using wrong
versions and I’d like to avoid that.
Examples:
zope2 via pip uses Aquisition 2.13.8
zope2 via ZMS3 installed via pip uses Aquisition 4.1
zope2 via pip uses ZopeUndo 2.12.0
zope2 via ZMS3 installed via pip uses ZopeUndo 4.0
Most packages with ZMS3 install are newer versions.
So, basic question: What is the recommended way to setup Zope2 with ZEO. Which
versions work? It seems almost impossible to understand what versions should be
used. The Zope2 egg uses way older versions. Any problems with that? Which
packages are safe to update?
Also, there were some patches by the ZMS people last year
(https://github.com/zopefoundation/Zope/pull/13) - but Zope2 installed via pip
works just fine here.
Using newer package versions require the patches though.
What is the „best“ setup if I’m using ZEO to connect with Zope2 ZEO clients AND
from other remote setups (non Zope-Clients, Python command line code working
with the ZODB via ClientStorage).
Maybe Tres could give me some insight on this?
Best,
Sebastian
Am 06.05.2015 um 07:42 schrieb Niels Dettenbach (Syndicat IT Internet)
n...@syndicat.com:
Am 6. Mai 2015 01:24:50 MESZ, schrieb Sebastian Tänzer s...@taenzer.me:
2015-05-06 01:16:46 WARNING ZEO.zrpc (19205) CW: error connecting to
('::1', ): ECONNREFUSED
2015-05-06 01:16:46 INFO ZEO.ClientStorage zeostorage Testing
connection ManagedClientConnection ('127.0.0.1', )
2015-05-06 01:16:46 INFO ZEO.zrpc.Connection('C') (127.0.0.1:)
hmm,
just a small shot in the dark: Even if this should work too - why the
client tries to connect the IPv4 address of loopback and the error gives the
IPv6 of it back?
I would try to eleminate this if possible and bring Zope down to IPv4
completely, because there are still many software libraries and applications
around (even some underlying os parts) which are not well tested with IPv6.
If this solves your prob - no idea...
hth a bit.
cheerioh,
Niels.
--
Niels Dettenbach
Syndicat IT Internet
http://www.syndicat.com
___
Zope maillist - Zope@zope.org
https://mail.zope.org/mailman/listinfo/zope
** No cross posts or HTML encoding! **
(Related lists -
https://mail.zope.org/mailman/listinfo/zope-announce
https://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] ZEO manageUndo not working - DisconnectedError
After further testing I was able to sort this one out by downgrading ZopeUndo
to 2.12.0 (pip install ZopeUndo==2.12.0 --force —upgrade).
PyPi only said 4.0 brought Py3 compability. Should I expect any problems using
ZopeUndo 2.12 + ZEO/ZODB4/ZODB3 with their latest versions?
Best, Sebastian
Am 06.05.2015 um 07:42 schrieb Niels Dettenbach (Syndicat IT Internet)
n...@syndicat.com:
Am 6. Mai 2015 01:24:50 MESZ, schrieb Sebastian Tänzer s...@taenzer.me:
2015-05-06 01:16:46 WARNING ZEO.zrpc (19205) CW: error connecting to
('::1', ): ECONNREFUSED
2015-05-06 01:16:46 INFO ZEO.ClientStorage zeostorage Testing
connection ManagedClientConnection ('127.0.0.1', )
2015-05-06 01:16:46 INFO ZEO.zrpc.Connection('C') (127.0.0.1:)
hmm,
just a small shot in the dark: Even if this should work too - why the
client tries to connect the IPv4 address of loopback and the error gives the
IPv6 of it back?
I would try to eleminate this if possible and bring Zope down to IPv4
completely, because there are still many software libraries and applications
around (even some underlying os parts) which are not well tested with IPv6.
If this solves your prob - no idea...
hth a bit.
cheerioh,
Niels.
--
Niels Dettenbach
Syndicat IT Internet
http://www.syndicat.com
___
Zope maillist - Zope@zope.org
https://mail.zope.org/mailman/listinfo/zope
** No cross posts or HTML encoding! **
(Related lists -
https://mail.zope.org/mailman/listinfo/zope-announce
https://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] ZEO manageUndo not working - DisconnectedError
Hi Sebastian, Am 06.05.2015 um 15:26 schrieb Sebastian Tänzer: After further testing I was able to sort this one out by downgrading ZopeUndo to 2.12.0 (pip install ZopeUndo==2.12.0 --force —upgrade). PyPi only said 4.0 brought Py3 compability. Should I expect any problems using ZopeUndo 2.12 + ZEO/ZODB4/ZODB3 with their latest versions? Undo has long been a standard functionality in Zope2 but proven to be quite dangerous as it is possible to totally mess up your object tree. IIRC undo has therefore been removed even from the ZODB - that is what you are experiencing with the current version. What seems to work quite well is to enable history support where you have access to every old transaction and can compare or copy them to be on top of the stack - which avoids the problem with disappearing transactions as Undo has. Unfortunately out of the box only a few Products have history support enabled. Some monkey patching can enable it for many of the other Products of stock Zope2. ObjectManager/Folder is a bit limited since accessing a whole historic version of a subtree can't be done easily. However, if you are interested and want to try: http://old.zope.org/Members/tino/PatchHistory/view (very old, but seems to work with the 2.12 at least) Best, Tino Best, Sebastian ___ Zope maillist - Zope@zope.org https://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] ZEO manageUndo not working - DisconnectedError
Am 6. Mai 2015 01:24:50 MESZ, schrieb Sebastian Tänzer s...@taenzer.me:
2015-05-06 01:16:46 WARNING ZEO.zrpc (19205) CW: error connecting to
('::1', ): ECONNREFUSED
2015-05-06 01:16:46 INFO ZEO.ClientStorage zeostorage Testing
connection ManagedClientConnection ('127.0.0.1', )
2015-05-06 01:16:46 INFO ZEO.zrpc.Connection('C') (127.0.0.1:)
hmm,
just a small shot in the dark: Even if this should work too - why the client
tries to connect the IPv4 address of loopback and the error gives the IPv6 of
it back?
I would try to eleminate this if possible and bring Zope down to IPv4
completely, because there are still many software libraries and applications
around (even some underlying os parts) which are not well tested with IPv6.
If this solves your prob - no idea...
hth a bit.
cheerioh,
Niels.
--
Niels Dettenbach
Syndicat IT Internet
http://www.syndicat.com
___
Zope maillist - Zope@zope.org
https://mail.zope.org/mailman/listinfo/zope
** No cross posts or HTML encoding! **
(Related lists -
https://mail.zope.org/mailman/listinfo/zope-announce
https://mail.zope.org/mailman/listinfo/zope-dev )
[Zope] Zeo scaling question
Hello, I am working on a solution that has a very high number of users and a significant amount of traffic. Using a zeo configuration we have been running into a few bottle necks while trying to improve our load testing results. The problem we see is if that put our Zeo Configuration under load that Zope does not close the connections and the application server ends up running out of connections for Apache to connect to. The connections once opened idle indefinitely. (Apache is running on a dedicated server different from our Zeo instances). We are using RR between 2 vm instances using 10 zeo clients per vm to distribute load. The bottleneck is occurring on the Apache server because it's keeping TCP connections in time_wait status. We have completed load tests on both Windows 2003 and 2008. In 2003 we were able to adjust the registry so the OS would terminate connections after 15 seconds of idle, however in 2008 the minimum is 30 seconds. In the upper levels of testing the OS runs out of TCP connections because it can't close them fast enough and it begins to fail requests. Ideally we don't want to close connections forcibly via the tcp stack because Zope keeps them open. We'd hope that Zope would manage this clean up gracefully. One option we are considering is using IIS7.5 and ARR as a replacement load balancing/rewrite method. This could allow us to check health of destinations prior to forwarding a request. It may also give us more control over closing connections at the OS level. One other detail we think might be the issue is that Zope is not initiating the close connection event. Does anyone have any experience or knowledge they can lend to help out? Configuration: Windows 2003/2008 server Apache 2.1 Zope 2.12 MS SQL 2005 Python 2.6.6 SQLAlchemy 0.6.5 sqlalchemy version is 0.6.5 and z3c.sqlalchemy=1.4.0 zope.sqlalchemy=0.6 Jimmy Small (mallaice) This electronic message is intended only for the use of the addressee(s) named above and may contain legally privileged and/or confidential information. If you are not the intended recipient of this message, you are notified that any dissemination, distribution or copying of this message is strictly prohibited. If you received this message in error, please immediately notify the sender by telephone and delete the original message. ___ Zope maillist - Zope@zope.org https://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope-dev )
[Zope] zeo cache-size tuning - number of obj. vs MB
Hi What's the difference between the to cache-size settings here? I've not been able to find a good description to guide me in finding the right values. zodb_db main mount-point / cache-size 1 1. zeoclient server foo:9080 storage main name zeostorage var $INSTANCE/var cache-size 20MB 2. /zeoclient /zodb_db As far as I've been able to determine, the second one 2. refers to a separate cache at the zeo connection level. Running 9 zeo clients (2.7.9) like this on a frontend with 4G ram, varnish, apache, and haproxy. Tethering on the edge of too much swapping. Needing to cut down.. Would it be better to spend all the memory on 1? Is there a good way to find a balance? Regards Gaute Amundsen ___ Zope maillist - Zope@zope.org https://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] zeo cache-size tuning - number of obj. vs MB
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Gaute Amundsen wrote: Hi What's the difference between the to cache-size settings here? I've not been able to find a good description to guide me in finding the right values. zodb_db main mount-point / cache-size 1 1. zeoclient server foo:9080 storage main name zeostorage var $INSTANCE/var cache-size 20MB 2. /zeoclient /zodb_db As far as I've been able to determine, the second one 2. refers to a separate cache at the zeo connection level. The ZEO client cache is a disk-based cache, separate from the RAM-based cache used by any ZODB connection. It doesn't use RAM, per se, except for an in-memory index of OID - (layer, offset). Running 9 zeo clients (2.7.9) like this on a frontend with 4G ram, varnish, apache, and haproxy. Tethering on the edge of too much swapping. Needing to cut down.. Assuming you are running with the default number of worker threads (4): 1 objects in the cache * 4 active connections per appserver * 9 appservers means you are caching 360,000 objects in RAM. If they are putting you into swap, then you are averaging something like 10Kb per object, which is pretty big. Would it be better to spend all the memory on 1? Is there a good way to find a balance? I think I might cut down on the number of appservers on that host, if it truly is hitting swap. Or trim the number of threads per appserver down from 4 to 2 or 3. Tres. - -- === Tres Seaver +1 540-429-0999 tsea...@palladion.com Palladion Software Excellence by Designhttp://palladion.com -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iEYEARECAAYFAktEUIQACgkQ+gerLs4ltQ6pYwCfX4g/ZVfAnyJNQ4Y1njnSx1HV 6QEAnAg/UHsCEQSx6rUpj5K9X2JkoGdQ =7ya0 -END PGP SIGNATURE- ___ Zope maillist - Zope@zope.org https://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - https://mail.zope.org/mailman/listinfo/zope-announce https://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] zeo cache-size tuning - number of obj. vs MB
On Wed, 2010-01-06 at 03:57 -0500, Tres Seaver wrote:
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
Gaute Amundsen wrote:
Hi
What's the difference between the to cache-size settings here?
I've not been able to find a good description to guide me in finding the
right values.
zodb_db main
mount-point /
cache-size 1 1.
zeoclient
server foo:9080
storage main
name zeostorage
var $INSTANCE/var
cache-size 20MB 2.
/zeoclient
/zodb_db
As far as I've been able to determine, the second one 2. refers to a
separate cache at the zeo connection level.
The ZEO client cache is a disk-based cache, separate from the RAM-based
cache used by any ZODB connection. It doesn't use RAM, per se, except
for an in-memory index of OID - (layer, offset).
What would happen if set too small or too large?
Running 9 zeo clients (2.7.9) like this on a frontend with 4G ram,
varnish, apache, and haproxy. Tethering on the edge of too much
swapping. Needing to cut down..
Assuming you are running with the default number of worker threads
(4): 1 objects in the cache * 4 active connections per appserver *
9 appservers means you are caching 360,000 objects in RAM. If they are
putting you into swap, then you are averaging something like 10Kb per
object, which is pretty big.
Sorry, forgot to mention: 1 thread per appserver.
Otherwise there's not much use in haproxy's load balancing we figured.
Meory usage as reported by munin is avg. 160M, max ~220M
and backend loads/stores are about 1000/10 a minute avg.
and iowait at 45% avg today.
committed memory 3.7G avg.
This little oneliner for summing VSZ:
ps auxf | grep zope | awk '{total+=$5} END{print total}'
reports:
1918552 for zope
1489768 for varnish
2270732 for www-data
And for RSS
1375908 zope
177200 varnsih
374624 www-data
in addition to the 9, that includes one appserver with 6 therads/6
obs and one with 1 therad/2 for developers and robots respectively.
Would it be better to spend all the memory on 1?
Is there a good way to find a balance?
I think I might cut down on the number of appservers on that host, if it
truly is hitting swap. Or trim the number of threads per appserver down
from 4 to 2 or 3.
Problem is we run out of them way to often now, and get 503's from
haproxy. I just added another 3 in fact..
Perhaps I'll try with 6 tomorrow..
Thanks :)
Gaute
___
Zope maillist - Zope@zope.org
https://mail.zope.org/mailman/listinfo/zope
** No cross posts or HTML encoding! **
(Related lists -
https://mail.zope.org/mailman/listinfo/zope-announce
https://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] zeo cache-size tuning - number of obj. vs MB
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
Gaute Amundsen wrote:
On Wed, 2010-01-06 at 03:57 -0500, Tres Seaver wrote:
The ZEO client cache is a disk-based cache, separate from the RAM-based
cache used by any ZODB connection. It doesn't use RAM, per se, except
for an in-memory index of OID - (layer, offset).
What would happen if set too small or too large?
If it is too small, your app will have to go across the net too often to
fetch an object's state from the ZEO server. If it is too large, then
recovering after the client disconnects from the ZEO server can take a
very long time.
Assuming you are running with the default number of worker threads
(4): 1 objects in the cache * 4 active connections per appserver *
9 appservers means you are caching 360,000 objects in RAM. If they are
putting you into swap, then you are averaging something like 10Kb per
object, which is pretty big.
Sorry, forgot to mention: 1 thread per appserver.
Otherwise there's not much use in haproxy's load balancing we figured.
I would probably test that assumption. One rule of thumb I use is one
appserver process per CPU, and then tune number of threads / connection
cache size until the machine stays just out of swap.
Meory usage as reported by munin is avg. 160M, max ~220M
and backend loads/stores are about 1000/10 a minute avg.
and iowait at 45% avg today.
committed memory 3.7G avg.
This little oneliner for summing VSZ:
ps auxf | grep zope | awk '{total+=$5} END{print total}'
reports:
1918552 for zope
1489768 for varnish
2270732 for www-data
And for RSS
1375908 zope
177200 varnsih
374624 www-data
in addition to the 9, that includes one appserver with 6 therads/6
obs and one with 1 therad/2 for developers and robots respectively.
Nothing jumps out at me: I might look at pushing varnish or apache off
onto another machine, maybe.
I think I might cut down on the number of appservers on that host, if it
truly is hitting swap. Or trim the number of threads per appserver down
from 4 to 2 or 3.
Problem is we run out of them way to often now, and get 503's from
haproxy. I just added another 3 in fact..
Perhaps I'll try with 6 tomorrow..
If your proxy is timing you out, then maybe you need to look at more
hardware, or elas spend some time profiling the application and trimming
out unnecessary work there.
Tres.
- --
===
Tres Seaver +1 540-429-0999 tsea...@palladion.com
Palladion Software Excellence by Designhttp://palladion.com
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
iEYEARECAAYFAktE7C8ACgkQ+gerLs4ltQ74QgCffLc5emNAK7l22qbdGDQgagVv
UnkAoMFwNe4cnO1grPe6x6aHt6WgdRNt
=sRPq
-END PGP SIGNATURE-
___
Zope maillist - Zope@zope.org
https://mail.zope.org/mailman/listinfo/zope
** No cross posts or HTML encoding! **
(Related lists -
https://mail.zope.org/mailman/listinfo/zope-announce
https://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] zeo client tracking
+---[ Mikael Kermorgant ]-- | Hello, | | We're using a zope 2.10 setup with a zeo server and several zeo clients behind | lvs. | | Sometimes, we'd like to track which zeo client handled teh request, and we're | going to need it as we'd like to proove that ha-proxy handles session | persistance. | | Is there a way to add a header in the http response or the html code to do that | ? Well the Zope Request has a bunch of things you can use; Adding in something into your template that stamps the page with it should be simple enough. e.g. div tal:content=request/HTTP_HOST style=display:none / -- Andrew Milton a...@theinternet.com.au ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
[Zope] ZEO configuration question ...
Hi All: I have a zeo setup which has one zeo server (say s1) and 5 zeo clients (say c1 through c5). All of these clients share a single Data.fs. Now, I need a zeo server (say s2) between just some of the zeo clients (say c1 and c2). all of these instances should share just one Dafa.fs file. Is this configuration possible/feasible. Any alternatives? Thanks for your time. -A ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] ZEO configuration question ...
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Analog Kid wrote: I have a zeo setup which has one zeo server (say s1) and 5 zeo clients (say c1 through c5). All of these clients share a single Data.fs. Now, I need a zeo server (say s2) between just some of the zeo clients (say c1 and c2). all of these instances should share just one Dafa.fs file. Is this configuration possible/feasible. Any alternatives? It is possible to mount a second storage at a given point within the main storage used by the database: the standard zope.conf, for instance, sets this up for the session data:: zodb_db temporary # Temporary storage database (for sessions) temporarystorage name temporary storage for sessioning /temporarystorage mount-point /temp_folder container-class Products.TemporaryFolder.TemporaryContainer /zodb_db You can do the same thing in your two special clients, setting up a mount for the private storage. One caveat is that you will have to add a mount point object (an instance of 'Products.ZODBMountPoint.MountedObject.MountedObject') in the main part of your database at the path pointed to by 'mount-point' in the configuration. The other clients will see the empty mount point, but won't be able to traverse into it, because they won't be configured with the stanza that actually creates the mount. HTH, Tres. - -- === Tres Seaver +1 540-429-0999 [EMAIL PROTECTED] Palladion Software Excellence by Designhttp://palladion.com -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.6 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org iD8DBQFI06Zz+gerLs4ltQ4RAiKgAKDQGU0vkSQsuSp9+3pTTrgtWcSSHACgvH0i +EDdtDrbsfoBQeQFmK9yJgs= =6rgr -END PGP SIGNATURE- ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] ZEO clusters on a single box
Jean Jordaan wrote at 2008-8-13 12:26 +0700: We have a server running many ZEO clusters (many more than the number of CPUs, of course). Each cluster consists of a master and two clients. Would it make sense to run both clients and spread requests across them? Usually, this would make sense when your clusters have very different request rates (such that the probability is high, that requests to a single cluster run in parallel while other clusters are lazy). -- Dieter ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] ZEO clusters on a single box
--On 13. August 2008 12:26:28 +0700 Jean Jordaan [EMAIL PROTECTED] wrote: Hi there We have a server running many ZEO clusters (many more than the number of CPUs, of course). Each cluster consists of a master and two clients. Would it make sense to run both clients and spread requests across them? Or would it be better to just increase the number of threads of a single client, and only use the second client for './bin/zopectl debug', and for failover if the first client needs to be restarted? Would it ever make sense to have *more* than 2 clients per master, in this scenario (many more processes than CPUs)? I think there is no general rule. E.g. we are running a production cluster with 10 ZEO clients where two or three nodes care about long-running requests (like document checkin/checkouts). All other clients deal with the standard request. With this approach we can have different configurations for the number of threads and cache-size. The balancing is done based on patterns where we identify the long-running requests through a pattern with the URL. -aj pgps19I5CQgcM.pgp Description: PGP signature ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
[Zope] ZEO clusters on a single box
Hi there We have a server running many ZEO clusters (many more than the number of CPUs, of course). Each cluster consists of a master and two clients. Would it make sense to run both clients and spread requests across them? Or would it be better to just increase the number of threads of a single client, and only use the second client for './bin/zopectl debug', and for failover if the first client needs to be restarted? Would it ever make sense to have *more* than 2 clients per master, in this scenario (many more processes than CPUs)? So far, I've had the following response to my questions: [...] at every zope/plone conference there are people who claim that adding more threads makes no difference and that you should rather add more zeo clients, but my CS background says more threads can share memory and perform better. But for zope it seems that each threads maintains it's own cache anyway, so there is little memory sharing going on. You may as well run 10 clients with one thread each rather than one client with 10 threads, it would make little difference other than the overhead for the extra process, which is less than 100MB. I'm a little disappointed in zope's memory usage patterns. It doesn't really matter how much memory you have or what sorts of limits you impose, it seems zope always uses more and more until it consumes about 1GB of swap and starts killing the box. So you need to monitor RAM usage with nagios or something and restart the instance every so often. On some of our managed machines we restart zope about once an hour because of this. That just doesn't make good sense, you kill your cache every time. RTFM with pointers most appreciated. -- jean . .. //\\\oo///\\ ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
[Zope] zeo configuration HOWTO
I'm trying to find more documentation about all the configuration parameters for a zeo.conf Namely, I have a zope with multiple zodb's and mount points. I am trying to learn how to migration a single zope instance with multiple mounts into zeo. One thing I don't understand is temporary storage. If I have zodb_db temporary # Temporary storage database (for sessions) temporarystorage name temporary storage for sessioning /temporarystorage mount-point /temp_folder container-class Products.TemporaryFolder.TemporaryContainer /zodb_db in zope.conf -- how do and transfer that into zeo.conf and modify the zope.conf to properly understand. specifically, do I put 'container-class' in zeo.conf ? Also, in zope.conf what do 'server', 'storage', 'name', and 'client' map to in zeo.conf? There must be a more detail descriptions of how to configure zeo somewhere. -- David Bear College of Public Programs at ASU 602-464-0424 ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] zeo configuration HOWTO
David Bear schrieb: I'm trying to find more documentation about all the configuration parameters for a zeo.conf I *believe* on plope.org there was an introduction to it with some more parameters explained. Namely, I have a zope with multiple zodb's and mount points. I am trying to learn how to migration a single zope instance with multiple mounts into zeo. One thing I don't understand is temporary storage. If I have zodb_db temporary # Temporary storage database (for sessions) temporarystorage name temporary storage for sessioning /temporarystorage mount-point /temp_folder container-class Products.TemporaryFolder.TemporaryContainer /zodb_db in zope.conf -- how do and transfer that into zeo.conf and modify the zope.conf to properly understand. this is where volatile thing are created and purged again. there is never anything to migrate from here specifically, do I put 'container-class' in zeo.conf ? there is a howto on this on either zope.org or (more probably) on plone.org Also, in zope.conf what do 'server', 'storage', 'name', and 'client' map to in zeo.conf? There must be a more detail descriptions of how to configure zeo somewhere. -- David Bear College of Public Programs at ASU 602-464-0424 ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev ) ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] zeo configuration HOWTO
--On 22. April 2008 15:16:55 -0700 David Bear [EMAIL PROTECTED] wrote: I'm trying to find more documentation about all the configuration parameters for a zeo.conf Namely, I have a zope with multiple zodb's and mount points. I am trying to learn how to migration a single zope instance with multiple mounts into zeo. One thing I don't understand is temporary storage. If I have zodb_db temporary # Temporary storage database (for sessions) temporarystorage name temporary storage for sessioning /temporarystorage mount-point /temp_folder container-class Products.TemporaryFolder.TemporaryContainer /zodb_db in zope.conf -- how do and transfer that into zeo.conf and modify the zope.conf to properly understand. specifically, do I put 'container-class' in zeo.conf ? Also, in zope.conf what do 'server', 'storage', 'name', and 'client' map to in zeo.conf? There must be a more detail descriptions of how to configure zeo somewhere. Check the documentation of the underlying ZConfig structure. I think ZEO/components.xml is a good start (check the spelling and search for something with *.xml :-)) -aj pgpePACKrKaGw.pgp Description: PGP signature ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Re: Zope Zeo Performance
FuBuJo wrote at 2008-3-14 22:06 +: ... I thought putting Apache in front of Zope was very common (using VirtualHost Monster) - guess not. This is common. But, usually, a ZEO client is not abbreviated as ZEO. ZEO usually means the ZEO server. ... It's the python process on the Zeo Client that grows large and seems to become unresponsive. Possibly due to the Transaction blocking being reported in the Zeo Severs log. Occational transaction blocking messages are no reason to worry about. Only, when you see it very often or the number of waiting clients is quite high, bad things are happening. -- Dieter ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Re: Zope Zeo Performance
FuBuJo schrieb: I apologize that my description was so confusing. I appreciate the feedback and so would like to clarify. The diagram is how the traffic flows. So we have: Step 1 - a Load Balancer that passes traffic to Apache Step 2 - Apache which uses mod_proxy to obfuscate the URL and proxies traffic to the Zeo Client is that so? does not apache talk to a zope server ? robert ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
[Zope] Zope Zeo Performance
Hello. I've thumbed through the list for Zope Zeo performance issues similar to mine - but have found a lot of conflicting information - so I thought I'd ask (and keep reading and experimenting). Zope - 2.7.x (Zeo) Apache 2.x Zope is using Plone (2.0.x) Basically I have a high traffic (high performance) Apache - Zeo - Zope(ZODB) setup. 10 Apache/Zeo boxes - and one big old Zope/ZODB box. The equipment is high end DELLs (well as of 3 years ago) - all running FreeBSD (we tried Red Hat but it blows up - but thats another post). Anywho - if your still reading thanks. We know Plone and code is a major issue - and we are working on a migration plan to 3. But it the meantime ... The traffic is heavy write traffic (I read some of Dieters posts and am testing that out as well). Once overall load hits about 100 people or so the Zeo's start dying - heavy load, slow response, python takes all CPU/Memory. Then when traffic is removed from the ZEO instance ... the system remains CPU bound by the python process ... and you have to bounce Zope(Zeo instance) and Apache to free it. The ZODB reports heavy Clients waiting ... but doesn't budge on load. So ... anyone have any suggestions. I can throw 10 more Apache/Zeo instances as it - but not sure if that's the right approach. So I guess here's my questions. 1. Is there a Zeo Client limit you can have when connecting to a Zope(Zeo Server) instance? 2. Are there any special setting to allow for 'many' Zeo clients connecting to Zeo server? 3. Are there any 'tweak' on the Zeo Client side or the Zeo Server side that I should consider? I've reviewed quite a bit on confirguration ... but nothing seems to really make a difference. 4. Anyone have any special 'high performance' tricks / tips they can point me to? I'll continue to read and research ... but any information would be helpful. Thanks -FuBuJo ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Zope Zeo Performance
FuBuJo wrote at 2008-3-14 13:31 +: You need to be a bit more careful in your description. For example the diagram Apache - Zeo - Zope(ZODB) is very confusing. It is very rare that Apache speaks to Zeo. The confusion between Zope and Zeo may go straight through your description such that it is often unclear whether you really mean Zope when you write Zope and Zeo when you write Zeo. More below. ... The traffic is heavy write traffic (I read some of Dieters posts and am testing that out as well). Once overall load hits about 100 people or so the Zeo's start dying Here again, you use a wrong word: dying would mean that your ZEO process terminates but below to say that it gets slower. - heavy load, slow response, python takes all CPU/Memory. Which python? The python executing Zeo? Or the one executing Zope? Then when traffic is removed from the ZEO instance ... the system remains CPU bound by the python process ... and you have to bounce Zope(Zeo instance) and Apache to free it. Which system? The one running ZEO (the ZEO server) or the one running Zope? The ZODB reports heavy Clients waiting ... but doesn't budge on load. You see this in the ZEO logfile? Then, it is ZEO which reports the waiting -- not the ZODB. So ... anyone have any suggestions. We are having similar problems -- I call them commit congestions. As far as we understand it by now, it is a multiple cause problem. Commit congestions can be caused on the client (=Zope) side and on the server (=ZEO) side. A client drastically increases the probability for commit congestions when he does expensive things while he helds the commit lock, i.e. during the second phase of the two phase commit protocol. We have identified three causes: * garbage collections During a garbage collection the garbage collector holds the GIL and blocks all Python activity. We found that a single generation 2 (i.e. full) garbage collection can take between 10 and 20 s. We had a bad text index implementation that caused excessive object creation and thereby lots of garbage collections. Our measure has been to drop the bad index implementation and reconfigure the garbage collector to reduce the garbage collection frequency by a factor of 1000 * stats in the second commit phase. In our system, stats for NFS served files could take up to 27 s. It is a complete mystery why. Local IO, too, occasionally seemed to need excessive time. This, too, is still mysterious. We may have some hints: some ranking bugs in a search engine could cause millions of IO operations within a short timeframe and may have significantly affected the Linux IO behaviour. * invalidation message reception and correspondng client cache updates during the second commit phase Other causes for commit contention come from the (Zeo) server: * FileStorage.pack unnecessarily holds the commit lock during large periods of the copying phase, drastically increasing the probability for commit contentions * during some pack phase (reachability analysis), access to the storage file is high volume and erratic. This drastically reduces the performance of the storage and make commit contentions likely. * other heavy use of the file system can affect the IO performance available for storage access and can increase the likelyhood for commit contentions. I can throw 10 more Apache/Zeo instances as it - but not sure if that's the right approach. It is not. Commit contention is a synchronization problem. It does not go away but is likely to increase when you scale your frontends up. So I guess here's my questions. 1. Is there a Zeo Client limit you can have when connecting to a Zope(Zeo Server) instance? There is no limit in principle -- but as you can see, lots of clients can affect performance. Invalidation message processing poses a load on the server which grows linearly with the number of clients (each client must get all invalidations). Most other Zeo load contributions are more dependent on the actual number of requested operations (reads, writes, commits) and less on the number of clients that request these operation (of cause, more clients can generated more requests). 2. Are there any special setting to allow for 'many' Zeo clients connecting to Zeo server? Reconfigure the Python garbage collector such that it runs far less often. Get rid of components that (unnecessarily) create lots of Python objects. Check whether you do unnecessary operations during the second commit phase. Place your ZODB storage files intelligently in the file system such that other high volume IO operations do not badly affect IO on the storage. -- Dieter ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org
[Zope] Re: Zope Zeo Performance
I apologize that my description was so confusing. I appreciate the feedback and so would like to clarify. The diagram is how the traffic flows. So we have: Step 1 - a Load Balancer that passes traffic to Apache Step 2 - Apache which uses mod_proxy to obfuscate the URL and proxies traffic to the Zeo Client Step 3 - the Zeo Client (residing on the same physical box as Apache) that then forwards the traffic to the Zeo Server Step 4- the Zeo Server that runs on its own box and writes to the ZODB. I thought putting Apache in front of Zope was very common (using VirtualHost Monster) - guess not. As for versions: FreeBSD 5.3 Python 2.3.5 Zope 2.7 The load on the Zeo Server is minuscule. The load on the Zeo Client(s) gets very large. It's the python process on the Zeo Client that grows large and seems to become unresponsive. Possibly due to the Transaction blocking being reported in the Zeo Severs log. Strangely disconnecting the Zeo Client from the Server while the Zeo Client is locked rarely gives the ... disconnected during transaction error message that I expected to see more of. Hope that clears up things for anyone else who'd like to chime in. Dieter - As for your suggestions. Thanks so much! I will certainly begin to investigate and pursue each. They seem very good things to do - just in general. I appreciate the input. I'll let the list knows how everything works out. I do notice that there was no mention of any general (config file) configuration techniques (increasing threads, cache objects, etc. etc.) --- is configuration not really that big of a performance boost? Thanks again for the direction. ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Zeo and conflict resolution (was suddenly confused)
Ricardo Newbery wrote at 2008-1-29 22:56 -0800: ... Another related question. With respect to conflict resolution, is is safe to assume that products (with _p_resolveConflict methods) in the main Zope products directory are already available to the ZEO server without any special configuration? Usually -- but of course not always. If you use and start ZEO from a Zope installation, then lib/python needs to be on PYTHONPATH such that ZEO finds its code and supporting packages. Then the content of lib/python/Products can be used normally as package Products. There is no magic needed for this. Of course, you can start ZEO from a non Zope installation (e.g. from a standalone ZEO installation) and then Zope's Products will be unavailable -- Dieter ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
[Zope] Zeo and conflict resolution (was suddenly confused)
On Jan 29, 2008, at 1:19 AM, Ricardo Newbery wrote: On Jan 19, 2008, at 10:39 AM, Dieter Maurer wrote: Zope uses Python import magic to make packages available in INSTANCE_HOME/Products as additional products in the Products package. If some of these products need application specific conflict resolution, the ZEO server needs to activate this import magic -- by importing Zope's App.FindHomes. -- Dieter Dieter, Apologies for jumping late in this thread but how precisely should App.FindHomes be imported? And a related question. Any suggestions on how to support multiple product directories like a non-zeo config? I checked already and zeo.conf doesn't appear to support the products directive. Ric Another related question. With respect to conflict resolution, is is safe to assume that products (with _p_resolveConflict methods) in the main Zope products directory are already available to the ZEO server without any special configuration? Ric ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
[Zope] Don't use Solaris for Zope/ZEO!
Occasionally, someone asks whether Solaris is an adequate platform for Zope/ZEO deployment. Up to now, I answered to those questions -- we are using Solaris in our backend and up to now we did not see problems. This changed a few days ago -- we met an incredible limitation in Solaris: the number of File streams is limited to 256 in 32 bit applications This limitation caused us ZEO restarts once or twice a day since several days (with a completely stupid error message EAGAIN from fdopen). The problem is described in http://blogs.sun.com/mandalika/entry/solaris_workaround_to_stdio_s Don't use Solaris for Zope/ZEO, unless your are willing to deploy it as a 64 bit application (which we will try with our ZEO) or you are using at least Solaris 10 update 4. Watch out for EAGAIN exceptions from fdopen or other file creation functions. They indicate that Solaris has hit its 256 File limit -- Dieter ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] ZEO clients going zombie
Bengt Giger wrote at 2007-4-10 15:36 +0200: with our new ZEO client systems, we frequently observe this problem: a ZEO process starts to use 100% CPU time (user) without a significant increase of requests. Sometimes (but not always) the process stops answering requests, still using 100% CPU. When we kill such a process, it changes to zombie state (shown in top as 'Z' and 'defunct'), still using 100% CPU, but now its system time, not user. The HTTP port is still in use, so we have to reboot this node to restart the ZEO client. This usually fails because some filesystems cannot be unmouted, there are still files locked. I tried both start modes, runzope and zopectl, but no difference. All that is in opposition of what I know about zombie processes, they should use no CPU time. I may have seen a similar problem (though I am not sure about the details): The problem was a buggy Python signal handling together with a doubtful Linux thread implementation (2.4 kernel). When a fatal signal occured, the signal killed the main thread but all other threads were in a strange state. Only a SIGKILL could get them out of this state. Of course, the ports remained open although nobody serviced them. I am not sure whether the threads were in zombie state or used CPU. Probably not. -- Dieter ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
[Zope] Zeo Client Hanging Unresponsive
I have posted this several times, but have not until now been able to
get DeadlockDebugger installed.
zope 2.9.5 + zeo
pythonm2.4.3
Red Hat RHEL 4
Plone 2.5.1
Our zeo clients hang intermittently. We have no way of reproducing the
problem, but it occurs daily. The client hangs and a restart seems to
fix the
problem.
In the event log with tracing on we get
Trace zeo.zrpc.Connection(C) wait(16697) {server:8100} pending, async=0
There are hundreds to thousands of these until the server is restarted.
In the zeo log we get
Error caught in asyncor asyncore.py
error:(110,'Connection timed out')
We have been trying to track this down and have had no luck. Does
anyone have
any suggestions? Below is our deadlock debugger output
Threads traceback dump at 2007-02-23 15:26:50
Thread -1269564496 (GET
/VirtualHostBase/https/soawds:443/VirtualHostRoot/Content///training):
File
/var/tmp/Zope-2.9.5-1-buildroot/apps1/zope2.9.5/lib/python/ZServer/PubC
ore/ZServerPublisher.py, line 23, in __init__
File
/var/tmp/Zope-2.9.5-1-buildroot/apps1/zope2.9.5/lib/python/ZPublisher/P
ublish.py, line 395, in publish_module
File
/var/tmp/Zope-2.9.5-1-buildroot/apps1/zope2.9.5/lib/python/ZPublisher/P
ublish.py, line 196, in publish_module_standard
File
/apps1/zope2.9.5/navo_instance/Products/PlacelessTranslationService/Pat
chStringIO.py, line 34, in new_publish
x = Publish.old_publish(request, module_name, after_list, debug)
File
/var/tmp/Zope-2.9.5-1-buildroot/apps1/zope2.9.5/lib/python/ZPublisher/P
ublish.py, line 115, in publish
File
/var/tmp/Zope-2.9.5-1-buildroot/apps1/zope2.9.5/lib/python/ZPublisher/m
apply.py, line 88, in mapply
File
/var/tmp/Zope-2.9.5-1-buildroot/apps1/zope2.9.5/lib/python/ZPublisher/P
ublish.py, line 41, in call_object
File
/var/tmp/Zope-2.9.5-1-buildroot/apps1/zope2.9.5/lib/python/Shared/DC/Sc
ripts/Bindings.py, line 311, in __call__
File
/var/tmp/Zope-2.9.5-1-buildroot/apps1/zope2.9.5/lib/python/Shared/DC/Sc
ripts/Bindings.py, line 348, in _bindAndExec
File
/apps1/zope2.9.5/navo_instance/Products/CMFCore/FSPageTemplate.py,
line 195, in _exec
result = self.pt_render(extra_context=bound_names)
File
/apps1/zope2.9.5/navo_instance/Products/CacheSetup/patch_cmf.py, line
38, in FSPT_pt_render
result = FSPageTemplate.inheritedAttribute('pt_render')(
File
/apps1/zope2.9.5/navo_instance/Products/CacheSetup/patch_cmf.py, line
92, in PT_pt_render
tal=not source, strictinsert=0)()
File
/var/tmp/Zope-2.9.5-1-buildroot/apps1/zope2.9.5/lib/python/TAL/TALInter
preter.py, line 238, in __call__
File
/var/tmp/Zope-2.9.5-1-buildroot/apps1/zope2.9.5/lib/python/TAL/TALInter
preter.py, line 281, in interpret
File
/var/tmp/Zope-2.9.5-1-buildroot/apps1/zope2.9.5/lib/python/TAL/TALInter
preter.py, line 749, in do_useMacro
File
/var/tmp/Zope-2.9.5-1-buildroot/apps1/zope2.9.5/lib/python/TAL/TALInter
preter.py, line 281, in interpret
File
/var/tmp/Zope-2.9.5-1-buildroot/apps1/zope2.9.5/lib/python/TAL/TALInter
preter.py, line 457, in do_optTag_tal
File
/var/tmp/Zope-2.9.5-1-buildroot/apps1/zope2.9.5/lib/python/TAL/TALInter
preter.py, line 442, in do_optTag
File
/var/tmp/Zope-2.9.5-1-buildroot/apps1/zope2.9.5/lib/python/TAL/TALInter
preter.py, line 437, in no_tag
File
/var/tmp/Zope-2.9.5-1-buildroot/apps1/zope2.9.5/lib/python/TAL/TALInter
preter.py, line 281, in interpret
File
/var/tmp/Zope-2.9.5-1-buildroot/apps1/zope2.9.5/lib/python/TAL/TALInter
preter.py, line 749, in do_useMacro
File
/var/tmp/Zope-2.9.5-1-buildroot/apps1/zope2.9.5/lib/python/TAL/TALInter
preter.py, line 281, in interpret
File
/var/tmp/Zope-2.9.5-1-buildroot/apps1/zope2.9.5/lib/python/TAL/TALInter
preter.py, line 507, in do_setLocal_tal
File
/var/tmp/Zope-2.9.5-1-buildroot/apps1/zope2.9.5/lib/python/Products/Pag
eTemplates/TALES.py, line 221, in evaluate
File
/var/tmp/Zope-2.9.5-1-buildroot/apps1/zope2.9.5/lib/python/Products/Pag
eTemplates/Expressions.py, line 185, in __call__
File
/var/tmp/Zope-2.9.5-1-buildroot/apps1/zope2.9.5/lib/python/Products/Pag
eTemplates/Expressions.py, line 180, in _eval
File
/var/tmp/Zope-2.9.5-1-buildroot/apps1/zope2.9.5/lib/python/Products/Pag
eTemplates/Expressions.py, line 85, in render
File
/apps1/zope2.9.5/navo_instance/Products/CMFPlone/browser/plone.py,
line 66, in globalize
self._initializeData(options=options)
File
/apps1/zope2.9.5/navo_instance/Products/CMFPlone/browser/plone.py,
line 147, in _initializeData
self._data['language'] = self.request.get('language', None) or \
File /apps1/zope2.9.5/navo_instance/Products/Archetypes/ClassGen.py,
line 58, in generatedAccessor
return schema[name].get(self, **kw)
File /apps1/zope2.9.5/navo_instance/Products/Archetypes/Field.py,
line 802, in get
value = ObjectField.get(self, instance, **kwargs)
File /apps1/zope2.9.5/navo_instance/Products/Archetypes/Field.py,
line 671, in get
return
[Zope] Re: zope/zeo on multiprocessor machines?
cristopher pierson ewing, on 2007-01-23: I've been told that zope is not set up to utilize the extra processing power of multiproc machines. I have a new intel x-serve with the dual-core dual-proc setup, and want to get the most I can out of zope. I've read a few articles that mention running more than one zope process as zeo clients sharing a single ZODB. What isn't mentioned is any additional seetup needed to ensure that each zeo client uses a separate processor. Is there any? Or is this automagically handled? A site we (Zest Software) made for a big customer runs on a dual-core server, with some multithreading so it seems like there are 4 processors. With one zeo client, only at most 25 percent of the processing power was used. We added a few zeo clients and saw the processor a lot more active, probably approaching 100 % when all clients are fully active. A big improvement, with no additional setup that I am aware of. So it is automagically handled, yes. -- Maurits van Rees | http://maurits.vanrees.org/ [NL] Work | http://zestsoftware.nl/ Do not worry about your difficulties in computers, I can assure you mine are still greater. ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
[Zope] zope/zeo on multiprocessor machines?
I've been told that zope is not set up to utilize the extra processing power of multiproc machines. I have a new intel x-serve with the dual-core dual-proc setup, and want to get the most I can out of zope. I've read a few articles that mention running more than one zope process as zeo clients sharing a single ZODB. What isn't mentioned is any additional seetup needed to ensure that each zeo client uses a separate processor. Is there any? Or is this automagically handled? Thanks in advance for any answers, Cris ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] zope/zeo on multiprocessor machines?
--On 23. Januar 2007 09:24:56 -0800 cristopher pierson ewing [EMAIL PROTECTED] wrote: I've been told that zope is not set up to utilize the extra processing power of multiproc machines. I have a new intel x-serve with the dual-core dual-proc setup, and want to get the most I can out of zope. I've read a few articles that mention running more than one zope process as zeo clients sharing a single ZODB. What isn't mentioned is any additional seetup needed to ensure that each zeo client uses a separate processor. You can use taskset on Linux (schedutils package). However it is somewhat a mistry about the overhead when you don't tie a process to a particular CPU. The current Linux scheduler tries to keep long-running processes on the same CPU instead of migrating them from one CPU to another one. The basic rule: 1 CPU per major process (ZEO client, ZEO server etc). -aj pgpyFsQZig8eC.pgp Description: PGP signature ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Zeo question
Ola, thx maciej and Jonathan...your answers have helped a lot Each zope instance can access multiple ZODB's (as Maciej mentions later), which means that the zope instance can access both a shared zodb and a local zodb which is specific to the zope instance. auaua...i didn't know that ;)...this is really cool. Now the inevitable question - 'Is there any documentation about how to set this up ?' Maybe I just define another zodb_db in my zope.conf and mount it into a folder like maciej said and just customize this folder. What i need to do is just create a custom skin on each zope instance to show that it is Manaus,Belem or wherever serving the information but have the same global content shown in each instance. From what you have said it seems this should be possible with a bit of tweaking Zope is very flexible! You just need to build a configuration which meets your use case! welcome to OSS, eh ! []'s -- Ian Lawrence http://ianlawrence.info Centre for Bioinformatics INSTITUTO NACIONAL DE PESQUISAS DA AMAZÔNIA-INPA RUA ANDRÉ ARAÚJO N º .2936 , BAIRRO DO ALEIXO MANAUS-AMAZONAS-BRAZIL Research Program in Biodiversity http://ppbio.inpa.gov.br PHONE: 055-92-3643-3358 CEP. 69011 -970 | Please do not send me documents in a closed | format.(*.doc,*.xls,*.ppt) | Use the open alternatives. (*.pdf,*.html,*.txt) http://www.gnu.org/philosophy/no-word-attachments.html return [type for type in types if type not in types_to_exclude] If you can see the beauty, then Python got you ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Zeo question
- Original Message - From: ian [EMAIL PROTECTED] To: Jonathan [EMAIL PROTECTED] Cc: Maciej Wisniowski [EMAIL PROTECTED]; zope@zope.org Sent: Wednesday, December 13, 2006 7:51 AM Subject: Re: [Zope] Zeo question Ola, thx maciej and Jonathan...your answers have helped a lot Each zope instance can access multiple ZODB's (as Maciej mentions later), which means that the zope instance can access both a shared zodb and a local zodb which is specific to the zope instance. auaua...i didn't know that ;)...this is really cool. Now the inevitable question - 'Is there any documentation about how to set this up ?' Maybe I just define another zodb_db in my zope.conf and mount it into a folder like maciej said and just customize this folder. To set up multiple mount points you need to edit your zope.conf (to configure the various mount points), then you need to Add 'ZODB Mount Point' via the ZMI at the appropriate location (note: this is for zope 2.7+). You can google for 'zodb mount points' if you need more info. What i need to do is just create a custom skin on each zope instance to show that it is Manaus,Belem or wherever serving the information but have the same global content shown in each instance. From what you have said it seems this should be possible with a bit of tweaking You can easily do this by changing the 'display' routines on each zope instance. Another method I have used is to set up URL subdomains for each 'different' entry point and then use Apache rewriting to automagically redirect the user to the correct location within the zope instance (ie. a different folder for each subdomain) - this allows you to have each zope instance be a real 'copy' (and not have to maintain different versions), but it depends on your use case as to which method is most appropriate for you! Jonathan ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Zeo question
auaua...i didn't know that ;)...this is really cool. Now the inevitable question - 'Is there any documentation about how to set this up ?' http://www.zope.org/Wikis/ZODB/HowtoRunAZEOServer/howto.html http://plope.com/Books/2_7Edition/ZEO.stx and one more interesting thing: http://longsleep.org/howto/sharesessionwithzeo/view -- Maciej Wisniowski ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Zeo question
Ola, You can easily do this by changing the 'display' routines on each zope instance I feel so stupid asking but what do you mean by this...i have googled but cannot find anything about 'zope display routines' ... Do you mean this: In the zope instances I can create a folder custom and place my customizations in there.. -- so I create logo_manaus.jpg for Manaus and this goes in the folder custom on the zope instance in Manaus and then i create logo_belem.jpg for Belem in the folder custom on the zope instance in Belem...and we live happily ever after ;) -- I think my problem is that i do not really understand what is stored in the ZODB...is it just content or is it content and configuration []'s -- Ian Lawrence http://ianlawrence.info Centre for Bioinformatics INSTITUTO NACIONAL DE PESQUISAS DA AMAZÔNIA-INPA RUA ANDRÉ ARAÚJO N º .2936 , BAIRRO DO ALEIXO MANAUS-AMAZONAS-BRAZIL Research Program in Biodiversity http://ppbio.inpa.gov.br PHONE: 055-92-3643-3358 CEP. 69011 -970 | Please do not send me documents in a closed | format.(*.doc,*.xls,*.ppt) | Use the open alternatives. (*.pdf,*.html,*.txt) http://www.gnu.org/philosophy/no-word-attachments.html return [type for type in types if type not in types_to_exclude] If you can see the beauty, then Python got you ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Zeo question
- Original Message - From: ian [EMAIL PROTECTED] To: Jonathan [EMAIL PROTECTED] Cc: zope@zope.org Sent: Wednesday, December 13, 2006 4:26 PM Subject: Re: [Zope] Zeo question Ola, You can easily do this by changing the 'display' routines on each zope instance I feel so stupid asking but what do you mean by this...i have googled but cannot find anything about 'zope display routines' ... Do you mean this: In the zope instances I can create a folder custom and place my customizations in there.. By 'display' routines I meant the page templates or dtml methods that you use to format your data/html. -- so I create logo_manaus.jpg for Manaus and this goes in the folder custom on the zope instance in Manaus and then i create logo_belem.jpg for Belem in the folder custom on the zope instance in Belem...and we live happily ever after ;) -- I think my problem is that i do not really understand what is stored in the ZODB...is it just content or is it content and configuration []'s You should have a read thru the zope book, i think this is the latest version: http://www.plope.com/Books/2_7Edition Jonathan ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
[Zope] Zeo question
Hi, Hope you can help 1. I am having trouble connecting a zope client to a zeo database.The server and client are in different goegraphical locations and behind different proxies and firewalls. I read that they communicate using a 'custom RPC protocol'...what does this mean and does it have any influence on which firewall/proxy rules we need to implement on our network?? 2. I can create a custom skin on each zope (in reality plone) instance right or is each zope instance is just a mirror of the other? thx -- Ian Lawrence http://ianlawrence.info Centre for Bioinformatics INSTITUTO NACIONAL DE PESQUISAS DA AMAZÔNIA-INPA RUA ANDRÉ ARAÚJO N º .2936 , BAIRRO DO ALEIXO MANAUS-AMAZONAS-BRAZIL Research Program in Biodiversity http://ppbio.inpa.gov.br PHONE: 055-92-3643-3358 CEP. 69011 -970 | Please do not send me documents in a closed | format.(*.doc,*.xls,*.ppt) | Use the open alternatives. (*.pdf,*.html,*.txt) http://www.gnu.org/philosophy/no-word-attachments.html return [type for type in types if type not in types_to_exclude] If you can see the beauty, then Python got you ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Zeo question
2. I can create a custom skin on each zope (in reality plone) instance right or is each zope instance is just a mirror of the other? Nope. It is something different. In general ZEO is about sharing ZODB (Zope Object Database) between few Zope instances. There is no mirroring but just one ZODB. ZODB is a place where all objects you have created through ZMI (Zope Management Interface) are hold. When you have standard setup (one Zope instance) it has it's own ZODB (you may see this in instance/var/data.fs. With ZEO you have ZEOServer that holds ZODB (data.fs file) and multiple ZEO Clients that are connecting to it. ZEOClient is a normal ZopeInstance that is not using it's own data.fs (ZODB) but shared one. Because of this, every object created/changed/removed on one ZEOClient is removed from shared ZODB therefore all ZEOClient see this. There is more things that can be done. You may for example define (in zope.conf) mount points for folders that are individual for each zope etc. -- Maciej Wisniowski ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Zeo question
- Original Message - From: Maciej Wisniowski [EMAIL PROTECTED] To: ian [EMAIL PROTECTED] Cc: zope@zope.org Sent: Tuesday, December 12, 2006 4:00 PM Subject: Re: [Zope] Zeo question 2. I can create a custom skin on each zope (in reality plone) instance right or is each zope instance is just a mirror of the other? Nope. It is something different. In general ZEO is about sharing ZODB (Zope Object Database) between few Zope instances. There is no mirroring but just one ZODB. ZODB is a place where all objects you have created through ZMI (Zope Management Interface) are hold. Scripts and external methods can also create objects which are stored in the ZODB When you have standard setup (one Zope instance) it has it's own ZODB (you may see this in instance/var/data.fs. With ZEO you have ZEOServer that holds ZODB (data.fs file) and multiple ZEO Clients that are connecting to it. ZEOClient is a normal ZopeInstance that is not using it's own data.fs (ZODB) but shared one. Each zope instance can access multiple ZODB's (as Maciej mentions later), which means that the zope instance can access both a shared zodb and a local zodb which is specific to the zope instance. Because of this, every object created/changed/removed on one ZEOClient is removed from shared ZODB therefore all ZEOClient see this. There is more things that can be done. You may for example define (in zope.conf) mount points for folders that are individual for each zope etc. -- Maciej Wisniowski You could store all objects in a shared zodb, even objects which are specific to a single zope instance (and control access via permissions, application logic, etc). And you can create objects in a local zodb (which would only be accessible by the local zope instance). Zope is very flexible! You just need to build a configuration which meets your use case! hth Jonathan ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Zeo Client hanging
On Wed, Oct 18, 2006 at 07:32:38AM +0200, Maciej Wisniowski wrote:
zope 2.8.5 + zeo
python 2.3.4 (red hat distribution)
Red Hat RHEL 4
Plone 2.1.2
Our zeo clients hang intermittently. We have no way of reproducing the
problem, but it occurs daily. The client hangs and a restart seems to
fix the
problem.
Install DeadlockDebugger and you'll be able to see what is
the state of your threads when Zope becomes unresponsive.
In the event log with tracing on we get
Trace zeo.zrpc.Connection(C) wait(16697) {server:8100} pending, async=0
There are hundreds to thousands of these until the server is restarted.
In the zeo log we get
Error caught in asyncore asyncore.py
error:(110,'Connection timed out')
I'm not sure what these errors are. Maybe firewall between zeo
server and zeo client closes the connections or something like that?
That was my guess too.
See this thread:
http://aspn.activestate.com/ASPN/Mail/Message/zope-list/2916870
--
Paul Winkler
http://www.slinkp.com
___
Zope maillist - Zope@zope.org
http://mail.zope.org/mailman/listinfo/zope
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Zeo Client hanging
I see references to asyncore.py near the end of the thread listing when
using DeadLockDebugger and seeing a stuck thread process. The tread
eventually clears out but sometimes an impatient user reloads, reloads,
reloads...and then all 4 threads are stuck on exact same thing. Then you
can see the same URL jump to the other Zope/ZEO client and same thing
happens there. Another few reloads and one user has effectively crashed
the siteuntil the threads clear themselves out. Usually is a catalog
search for a news article or something spanning our 6 years of online
news articles searching on the word 'bob' or something like that.
+1 for DeadlockDebugger... love it.
Allen
Paul Winkler wrote:
On Wed, Oct 18, 2006 at 07:32:38AM +0200, Maciej Wisniowski wrote:
zope 2.8.5 + zeo
python 2.3.4 (red hat distribution)
Red Hat RHEL 4
Plone 2.1.2
Our zeo clients hang intermittently. We have no way of reproducing the
problem, but it occurs daily. The client hangs and a restart seems to
fix the
problem.
Install DeadlockDebugger and you'll be able to see what is
the state of your threads when Zope becomes unresponsive.
In the event log with tracing on we get
Trace zeo.zrpc.Connection(C) wait(16697) {server:8100} pending, async=0
There are hundreds to thousands of these until the server is restarted.
In the zeo log we get
Error caught in asyncore asyncore.py
error:(110,'Connection timed out')
I'm not sure what these errors are. Maybe firewall between zeo
server and zeo client closes the connections or something like that?
That was my guess too.
See this thread:
http://aspn.activestate.com/ASPN/Mail/Message/zope-list/2916870
___
Zope maillist - Zope@zope.org
http://mail.zope.org/mailman/listinfo/zope
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope-dev )
[Zope] Zeo Client Unresponsive
Hello,
We have been having problem with our production servers. We currently have:
zope 2.8.5 + zeo
pythonm2.3.4
Red Hat RHEL 4
Plone 2.1.2
Our zeo clients hang intermittently. We have no way of reproducing the
problem, but it occurs daily. The client hangs and a restart seems to fix the
problem.
In the event log with tracing on we get
Trace zeo.zrpc.Connection(C) wait(16697) {server:8100} pending, async=0
There are hundreds to thousands of these until the server is restarted.
In the zeo log we get
Error caught in asyncor asyncore.py
error:(110,'Connection timed out')
We have been trying to track this down and have had no luck. Does anyone have
any suggestions?
Thanks,
Paul
___
Zope maillist - Zope@zope.org
http://mail.zope.org/mailman/listinfo/zope
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope-dev )
[Zope] Zeo Client hanging
Hello,We have been having problem with our production servers. We currently have:zope 2.8.5 + zeopython 2.3.4 (red hat distribution)Red Hat RHEL 4Plone 2.1.2Our zeo clients hang intermittently. We have no way of reproducing the problem, but it occurs daily. The client hangs and a restart seems to fix the problem.In the event log with tracing on we getTrace zeo.zrpc.Connection(C) wait(16697) {server:8100} pending, async=0There are hundreds to thousands of these until the server is restarted.In the zeo log we getError caught in asyncore asyncore.pyerror:(110,'Connection timed out')We have been trying to track this down and have had no luck. Does anyone have any suggestions?Thanks,Paul
___
Zope maillist - Zope@zope.org
http://mail.zope.org/mailman/listinfo/zope
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Zeo Client hanging
zope 2.8.5 + zeo
python 2.3.4 (red hat distribution)
Red Hat RHEL 4
Plone 2.1.2
Our zeo clients hang intermittently. We have no way of reproducing the
problem, but it occurs daily. The client hangs and a restart seems to
fix the
problem.
Install DeadlockDebugger and you'll be able to see what is
the state of your threads when Zope becomes unresponsive.
In the event log with tracing on we get
Trace zeo.zrpc.Connection(C) wait(16697) {server:8100} pending, async=0
There are hundreds to thousands of these until the server is restarted.
In the zeo log we get
Error caught in asyncore asyncore.py
error:(110,'Connection timed out')
I'm not sure what these errors are. Maybe firewall between zeo
server and zeo client closes the connections or something like that?
--
Maciej Wisniowski
___
Zope maillist - Zope@zope.org
http://mail.zope.org/mailman/listinfo/zope
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] ZEO and Data.fs
Title: Re: [Zope] ZEO and Data.fs No, you just need to change zope.conf to move from FileStorage to ClientStorage, and then you can move your Data.fs file to ZEO var directory and start zeo. Pascal De : Sinang, Danny [EMAIL PROTECTED] Date : Mon, 18 Sep 2006 21:38:48 +0800 : zope@zope.org Conversation : ZEO and Data.fs Objet : [Zope] ZEO and Data.fs Hello, We're currently not using ZEO, but may do so in the future. Question is, do we need to export or convert our Data.fs to a ZEO-capable format ? Regards, Danny ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev ) ** This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you have received this email in error please notify the system manager. This footnote also confirms that this email message has been swept by MIMEsweeper for the presence of computer viruses. www.mimesweeper.com ** ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
[Zope] ZEO and Data.fs
Hello, We're currently not using ZEO, but may do so in the future. Question is, do we need to export or convertour Data.fs toa ZEO-capableformat ? Regards, Danny ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] ZEO and Data.fs
--On 18. September 2006 21:38:48 +0800 Sinang, Danny [EMAIL PROTECTED] wrote: Hello, We're currently not using ZEO, but may do so in the future. Question is, do we need to export or convert our Data.fs to a ZEO-capable format ? No, just create a new ZEO instance and copy over the Data.fs from the existing standalone installation...easy, isn't it? :-) -aj pgpBhZpi3onV9.pgp Description: PGP signature ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] ZEO with Multiple Storages
On 9/14/06, Jonathan [EMAIL PROTECTED] wrote: If you mean having zope commit a single transaction to multiple storages, then 'vanilla' zope won't do it (Zope Corp has a commercial product called ZRS which does this type of thing). No, that's not true. ZRS is used to create hot read-only or stand-by copies of a ZEO server (and all storages within such a server are copied). One ZEO server can deal with multiple storages just fine. Transactions involving multiple storages commit just fine. First off, does this make any sense? Is it worth pursuing? I think so. I would think, but do not know for sure, that transactions only involving one storage should not delay transactions involving other storages. Secondly, I assume that where an object is stored does not limit what other objects it can access in a transaction (e.g. object A could add a property to object B if they were in different storages) because all changes would occur in a client. (this ties into the single transaction committing to multiple storages) You assume correctly. Note that a stock Zope install already runs multiple storages; one for Session data (tempstorage) and one for all other data. Lastly, any thoughts on how we could split up our existing data.fs file? One thought I had was to attempt to import/export data. Either before or after upgrading to 2.9 (probably best after to take advantage of the more robust ZODB), export and import. -- Martijn Pieters ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] ZEO with Multiple Storages
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 14 Sep 2006, at 08:54, Martijn Pieters wrote: One ZEO server can deal with multiple storages just fine. Transactions involving multiple storages commit just fine. Caveat: You buy nothing by having a single ZEO process serve several ZODBs when you're trying to decrease commit times. You should run a ZEO process per ZODB you're serving out, that way the writes can be segregated and parallelized. That also makes it easier to move processes/databases around disks/hosts to spread I/O load. jens -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.1 (Darwin) iD8DBQFFCQSiRAx5nvEhZLIRAhihAKC7V0lCz3deSz0vN/0RhvdWmFAj0ACgoguT I8NDYcUbz5t25xniZZ1akfo= =63My -END PGP SIGNATURE- ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] ZEO with Multiple Storages
- Original Message - From: Martijn Pieters [EMAIL PROTECTED] To: Jonathan [EMAIL PROTECTED] Cc: Brian Brinegar [EMAIL PROTECTED]; zope@zope.org Sent: Thursday, September 14, 2006 2:54 AM Subject: Re: [Zope] ZEO with Multiple Storages On 9/14/06, Jonathan [EMAIL PROTECTED] wrote: If you mean having zope commit a single transaction to multiple storages, then 'vanilla' zope won't do it (Zope Corp has a commercial product called ZRS which does this type of thing). No, that's not true. ZRS is used to create hot read-only or stand-by copies of a ZEO server (and all storages within such a server are copied). One ZEO server can deal with multiple storages just fine. Transactions involving multiple storages commit just fine. Yes and No! Yes: a single zeo server can deal with multiple storages (ie. a storage can contain mount points to other storages) Yes: if a single transaction creates/modifies multiple objects and each of those objects is stored in a different storage (all storages 'served up' by a single zeo server), then the single transaction will commit those objects to their respective storages with no problem. (however, there are little/no performance improvements in this scenario) No: if a single transaction creates/modifies a single object, that single object can not be stored over multiple storages (ie. you cannot achieve RAID 'disk striping' type of performance improvements) However, that being said there may be some improvement, if the application is disk-bound, by giving each storage its own disk subsystem (never tried this myself, but it may buy some small performance improvement). For the cost/effort involved I would stick to having a single zeo server 'serve up' a single storage, and have each zeo server on its own hardware platform (much more scalable, easy to implement). Jonathan ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] ZEO with Multiple Storages
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 14 Sep 2006, at 13:56, Jonathan wrote: Yes: a single zeo server can deal with multiple storages (ie. a storage can contain mount points to other storages) That's misleading wording, a storage knows nothing at all about mounts. The storage server can't do anything except for reading from and writing to ZODB databases. Mounting is a client concept only. jens -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.1 (Darwin) iD8DBQFFCVdQRAx5nvEhZLIRAkACAJwM7Cv0+dmnjgtsGsnYhYb6Vzl8BwCgqO8Y z4WxwoWO/XOrRkW8FwLZx9k= =4Lax -END PGP SIGNATURE- ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] ZEO with Multiple Storages
- Original Message - From: Jens Vagelpohl [EMAIL PROTECTED] To: [Zope] List Mailing zope@zope.org Sent: Thursday, September 14, 2006 9:21 AM Subject: Re: [Zope] ZEO with Multiple Storages -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 14 Sep 2006, at 13:56, Jonathan wrote: Yes: a single zeo server can deal with multiple storages (ie. a storage can contain mount points to other storages) That's misleading wording, a storage knows nothing at all about mounts. The storage server can't do anything except for reading from and writing to ZODB databases. Mounting is a client concept only. P.S. Or is the mount point totally controlled by zope.conf and nothing relating to the 'mount point' is stored in the storage? Jonathan ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] ZEO with Multiple Storages
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 14 Sep 2006, at 15:34, Jonathan wrote: Yes: a single zeo server can deal with multiple storages (ie. a storage can contain mount points to other storages) That's misleading wording, a storage knows nothing at all about mounts. The storage server can't do anything except for reading from and writing to ZODB databases. Mounting is a client concept only. P.S. Or is the mount point totally controlled by zope.conf and nothing relating to the 'mount point' is stored in the storage? You got it. jens -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.1 (Darwin) iD8DBQFFCV48RAx5nvEhZLIRAsLWAKCIYC9D+/QpX6oq3QvHvWSBtMMCGQCfY5fS LLsShBUeXJI+O6FSZRiYqxw= =/czT -END PGP SIGNATURE- ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] ZEO with Multiple Storages
- Original Message - From: Jens Vagelpohl [EMAIL PROTECTED] To: [Zope] List Mailing zope@zope.org Sent: Thursday, September 14, 2006 9:21 AM Subject: Re: [Zope] ZEO with Multiple Storages -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 14 Sep 2006, at 13:56, Jonathan wrote: Yes: a single zeo server can deal with multiple storages (ie. a storage can contain mount points to other storages) That's misleading wording, a storage knows nothing at all about mounts. The storage server can't do anything except for reading from and writing to ZODB databases. Mounting is a client concept only. I didn't say that the storage 'knew' anything about mount points. I simply stated that a storage can contain mount points to other storages (storages contain objects, if one of those objects happens to be a mount point to another storage the storage containing the 'mount point' object doesn't care). So the wording is not misleading (if the storage doesn't contain the mount point object, where is it stored?) Jonathan ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] ZEO with Multiple Storages
Brian Brinegar wrote at 2006-9-13 21:22 -0400: ... partioning large data sets onto different storages ... First off, does this make any sense? I makes sense and we do it successfully with our large editorial system. However, ensure that each storage is self contained data wise: e.g. do not put data in one storage and have them catalogued in a different one. Otherwise, selective backup of individual storages will introduce inconsistencies. Is it worth pursuing? Yes. Secondly, I assume that where an object is stored does not limit what other objects it can access in a transaction (e.g. object A could add a property to object B if they were in different storages) because all changes would occur in a client. (this ties into the single transaction committing to multiple storages) Right. -- Dieter ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] ZEO with Multiple Storages
Jens Vagelpohl wrote at 2006-9-14 15:50 +0200: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 14 Sep 2006, at 15:34, Jonathan wrote: Yes: a single zeo server can deal with multiple storages (ie. a storage can contain mount points to other storages) That's misleading wording, a storage knows nothing at all about mounts. The storage server can't do anything except for reading from and writing to ZODB databases. Mounting is a client concept only. P.S. Or is the mount point totally controlled by zope.conf and nothing relating to the 'mount point' is stored in the storage? You got it. I am not sure about the most recent Zope/ZODB versions. But, up to Zope 2.8/ZODB 3.4, Jonathan was right: Beside configuration, there have been MountPoint objects in the ZODB that handled the mounting magic. While not completely impossible, I would be very surprised if the need for MountPoint objects were dropped in newer Zope versions... -- Dieter ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
[Zope] ZEO with Multiple Storages
We currently have a ZEO environment with FileStorage (data.fs) over 100 gigs, which continues to grow. Currently we are approaching a major hardware and software upgrade. All new hardware and moving from Zope 2.6 to Zope 2.9. Our server hosts sites for several different schools and departments within our college, what I have considered doing is trying to split up the data.fs into multiple smaller storages, possibly one per school. I suspect if this is possible it could potentially have several benefits. The first being that we would not have to deal with 100 gig files which are very difficult to move around backup/restore, etc. Secondly, we often have a very large number of users updating content at a given time and will run into slowdowns where a transaction is blocked while another commits. I hope that multiple storages could allow for a transaction to commit to one storage at the same time as another, though I am not sure. Along this line can a single transaction commit to multiple storages? First off, does this make any sense? Is it worth pursuing? Secondly, I assume that where an object is stored does not limit what other objects it can access in a transaction (e.g. object A could add a property to object B if they were in different storages) because all changes would occur in a client. (this ties into the single transaction committing to multiple storages) If this does makes sense is there any documentation, recommendations, best practices for how to set something like this up? Lastly, any thoughts on how we could split up our existing data.fs file? One thought I had was to attempt to import/export data. Thank you, Brian Brinegar Web Systems Developer Engineering Computer Network Purdue University ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] ZEO with Multiple Storages
- Original Message - From: Brian Brinegar [EMAIL PROTECTED] To: zope@zope.org Sent: Wednesday, September 13, 2006 9:22 PM Subject: [Zope] ZEO with Multiple Storages We currently have a ZEO environment with FileStorage (data.fs) over 100 gigs, which continues to grow. Currently we are approaching a major hardware and software upgrade. All new hardware and moving from Zope 2.6 to Zope 2.9. Our server hosts sites for several different schools and departments within our college, what I have considered doing is trying to split up the data.fs into multiple smaller storages, possibly one per school. I suspect if this is possible it could potentially have several benefits. The first being that we would not have to deal with 100 gig files which are very difficult to move around backup/restore, etc. Secondly, we often have a very large number of users updating content at a given time and will run into slowdowns where a transaction is blocked while another commits. I hope that multiple storages could allow for a transaction to commit to one storage at the same time as another, though I am not sure. Along this line can a single transaction commit to multiple storages? If you mean having zope commit a single transaction to multiple storages, then 'vanilla' zope won't do it (Zope Corp has a commercial product called ZRS which does this type of thing). Alternatively, if your server is disk-bound then you could look into RAID disk striping to try to improve performance. First off, does this make any sense? Is it worth pursuing? Secondly, I assume that where an object is stored does not limit what other objects it can access in a transaction (e.g. object A could add a property to object B if they were in different storages) because all changes would occur in a client. (this ties into the single transaction committing to multiple storages) If this does makes sense is there any documentation, recommendations, best practices for how to set something like this up? Lastly, any thoughts on how we could split up our existing data.fs file? One thought I had was to attempt to import/export data. If you have the time and hardware you could easily set up several zeo servers on different 'back-end' hardware servers (ie. put each zeo server on its on computer) to improve performance, increase reliability/robustness, etc. Your 'use case' in which you have different schools and departments would lend itself to this kind of distribution (put the biggest/heaviest users on their own servers, group together smaller/lighter users on shared servers). You just need to define a mount point (in zope.conf) for each different zeo server (and set up each zeo server of course). If you already have the data for the various schools depts stored in separate folders, it would be easy to export each 'school' folder and then import it into the new configuration. hth Jonathan ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
[Zope] ZEO Problem
Hi; I'm reading the Plone book. It states that for Zope 2.7 (I have 2.78) ZEO is built in by default but must be instantiated thus: cd /opt/Zope-2/7/bin ./mkzeoinstance /var/zeo or some such code. So I tried this: cd /usr/local/zope/278/bin ./mkzeoinstance.py ../../instance2/var/zeo where 278 is my installation, instance2 is a Zope instance (I have 3), and mkzeoinstance.py is the command that works (mkzeoinstance by itself doesn't print anything to screen other than a command failed message). The Plone book then states that I must move Data.fs into the new /zeo/var dir, which I did. Everything seems to go well. If I start Zope using runzope I get this output at the tail end: -- 2006-09-02T13:15:35 INFO(0) Zope Ready to handle requests However, if I try to log into the ZMI my passwords don't work. I even tried creating an emergencyuser and that didn't work, either. What to do? Also, should I move these other files into the new zeo/var dir? Data.fs.index Data.fs.lock Data.fs.old Data.fs.tmp I tried that, also, with negative results, but it seems to me that if I'm moving Data.fs I should move these others as well. TIA, beno ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] ZEO Problem
- Original Message -
From: beno [EMAIL PROTECTED]
To: zope@zope.org
Sent: Saturday, September 02, 2006 9:24 AM
Subject: [Zope] ZEO Problem
Hi;
I'm reading the Plone book. It states that for Zope 2.7 (I have 2.78) ZEO
is built in by default but must be instantiated thus:
cd /opt/Zope-2/7/bin
./mkzeoinstance /var/zeo
or some such code. So I tried this:
cd /usr/local/zope/278/bin
./mkzeoinstance.py ../../instance2/var/zeo
where 278 is my installation, instance2 is a Zope instance (I have 3), and
mkzeoinstance.py is the command that works (mkzeoinstance by itself
doesn't print anything to screen other than a command failed message). The
Plone book then states that I must move Data.fs into the new /zeo/var dir,
which I did. Everything seems to go well. If I start Zope using runzope I
get this output at the tail end:
--
2006-09-02T13:15:35 INFO(0) Zope Ready to handle requests
In your .../log/events.log file you should see some zeo connection activity,
like:
2006-09-02T08:11:17 INFO Zope Ready to handle requests
--
2006-09-02T08:11:44 INFO ZEO.ClientStorage (3196) ClientStorage (pid=3196)
created RW/normal for storage: '1'
--
2006-09-02T08:11:44 INFO ZEO.cache created temporary cache file 'fdopen'
--
2006-09-02T08:11:44 INFO ZEO.ClientStorage (3196) Testing connection
ManagedClientConnection ('194.164.124.4', 8100)
--
2006-09-02T08:11:44 INFO ZEO.zrpc.Connection(C) (194.164.124.4:8100)
received handshake 'Z303'
--
2006-09-02T08:11:44 INFO ZEO.ClientStorage (3196) Server authentication
protocol None
--
2006-09-02T08:11:44 INFO ZEO.ClientStorage (3196) Connected to storage:
('sparrow.conetra.com', 8100)
--
2006-09-02T08:11:44 INFO ZEO.ClientStorage (3196) Verifying cache
--
2006-09-02T08:11:44 INFO ZEO.ClientStorage (3196) Waiting for cache
verification to finish
--
2006-09-02T08:11:44 INFO ZEO.ClientStorage (3196) Waiting for cache
verification to finish
--
2006-09-02T08:11:44 INFO ZEO.ClientStorage (3196) endVerify finishing
--
2006-09-02T08:11:44 INFO ZEO.ClientStorage (3196) endVerify finished
--
This will tell you that your zeo client is connected to the zeo server
However, if I try to log into the ZMI my passwords don't work. I even
tried creating an emergencyuser and that didn't work, either. What to do?
Also, should I move these other files into the new zeo/var dir?
Data.fs.index Data.fs.lockData.fs.old
Data.fs.tmp
All you need is Data.fs.
Data.fs.index will be recreated if it is not found when zope starts
Data.fs.old is created when you pack your zodb
You can safely ignore the .lock and .tmp files (they are created by Zope as
needed)
Jonathan
___
Zope maillist - Zope@zope.org
http://mail.zope.org/mailman/listinfo/zope
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] ZEO Problem
I got this working by changing the following in zope.conf:
server localhost:7080
to
server 202.71.106.119:
Apparently, it needs its own port?
TIA,
beno
Jonathan wrote:
- Original Message - From: beno [EMAIL PROTECTED]
To: zope@zope.org
Sent: Saturday, September 02, 2006 9:24 AM
Subject: [Zope] ZEO Problem
Hi;
I'm reading the Plone book. It states that for Zope 2.7 (I have 2.78)
ZEO is built in by default but must be instantiated thus:
cd /opt/Zope-2/7/bin
./mkzeoinstance /var/zeo
or some such code. So I tried this:
cd /usr/local/zope/278/bin
./mkzeoinstance.py ../../instance2/var/zeo
where 278 is my installation, instance2 is a Zope instance (I have
3), and mkzeoinstance.py is the command that works (mkzeoinstance by
itself doesn't print anything to screen other than a command failed
message). The Plone book then states that I must move Data.fs into
the new /zeo/var dir, which I did. Everything seems to go well. If I
start Zope using runzope I get this output at the tail end:
--
2006-09-02T13:15:35 INFO(0) Zope Ready to handle requests
In your .../log/events.log file you should see some zeo connection
activity, like:
2006-09-02T08:11:17 INFO Zope Ready to handle requests
--
2006-09-02T08:11:44 INFO ZEO.ClientStorage (3196) ClientStorage
(pid=3196) created RW/normal for storage: '1'
--
2006-09-02T08:11:44 INFO ZEO.cache created temporary cache file
'fdopen'
--
2006-09-02T08:11:44 INFO ZEO.ClientStorage (3196) Testing connection
ManagedClientConnection ('194.164.124.4', 8100)
--
2006-09-02T08:11:44 INFO ZEO.zrpc.Connection(C) (194.164.124.4:8100)
received handshake 'Z303'
--
2006-09-02T08:11:44 INFO ZEO.ClientStorage (3196) Server
authentication protocol None
--
2006-09-02T08:11:44 INFO ZEO.ClientStorage (3196) Connected to
storage: ('sparrow.conetra.com', 8100)
--
2006-09-02T08:11:44 INFO ZEO.ClientStorage (3196) Verifying cache
--
2006-09-02T08:11:44 INFO ZEO.ClientStorage (3196) Waiting for cache
verification to finish
--
2006-09-02T08:11:44 INFO ZEO.ClientStorage (3196) Waiting for cache
verification to finish
--
2006-09-02T08:11:44 INFO ZEO.ClientStorage (3196) endVerify finishing
--
2006-09-02T08:11:44 INFO ZEO.ClientStorage (3196) endVerify finished
--
This will tell you that your zeo client is connected to the zeo server
However, if I try to log into the ZMI my passwords don't work. I even
tried creating an emergencyuser and that didn't work, either. What to
do? Also, should I move these other files into the new zeo/var dir?
Data.fs.index Data.fs.lockData.fs.old
Data.fs.tmp
All you need is Data.fs.
Data.fs.index will be recreated if it is not found when zope starts
Data.fs.old is created when you pack your zodb
You can safely ignore the .lock and .tmp files (they are created by
Zope as needed)
Jonathan
___
Zope maillist - Zope@zope.org
http://mail.zope.org/mailman/listinfo/zope
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] ZEO Problem
- Original Message - From: beno [EMAIL PROTECTED] To: zope@zope.org Sent: Saturday, September 02, 2006 10:27 AM Subject: Re: [Zope] ZEO Problem I got this working by changing the following in zope.conf: server localhost:7080 to server 202.71.106.119: Apparently, it needs its own port? TIA, beno You need to make sure that the entry in your zeo client config file (zope.conf) eg zeoclient server 194.123.123.123:8100 matches the entry in your zeo server config file (zeo.conf) eg. zeo address 8100 and that your zeo server is running on the ip address specified in the zeoclient entry in zope.conf Jonathan ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] ZEO Problem
Jonathan wrote: I got this working by changing the following in zope.conf: server localhost:7080 to server 202.71.106.119: Apparently, it needs its own port? TIA, beno You need to make sure that the entry in your zeo client config file (zope.conf) eg zeoclient server 194.123.123.123:8100 matches the entry in your zeo server config file (zeo.conf) eg. zeo address 8100 and that your zeo server is running on the ip address specified in the zeoclient entry in zope.conf That's logical. Why does zeo need its own port? ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] ZEO Problem
--On 2. September 2006 12:15:47 -0400 beno [EMAIL PROTECTED] wrote: zeo address 8100 and that your zeo server is running on the ip address specified in the zeoclient entry in zope.conf That's logical. Why does zeo need its own port? Because every kind of network service needs its own port. -aj -- ZOPYX Ltd. Co. KG - Charlottenstr. 37/1 - 72070 Tübingen - Germany Web: www.zopyx.com - Email: [EMAIL PROTECTED] - Phone +49 - 7071 - 793376 E-Publishing, Python, Zope Plone development, Consulting pgpZ1fCZcP7pN.pgp Description: PGP signature ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] ZEO Problem Clients Waiting
On Mon, Aug 07, 2006 at 09:09:56PM +0200, Dieter Maurer wrote:
Gerhard Schmidt wrote at 2006-8-7 15:54 +0200:
...
2006-08-07T14:29:19 INFO ZEO.StorageServer (97002/10.152.64.23:52518)
Transaction
+blocked waiting for storage. Clients waiting: 1.
...
2006-08-07T14:29:50 INFO ZEO.StorageServer (97002/10.152.64.17:54463)
Blocked transaction restarted. Clients waiting: 1
2006-08-07T14:29:50 INFO ZEO.StorageServer (97002/10.152.64.23:52518)
Blocked transaction restarted.
This one was a very quick one only 30 seconds. I have Blocked Transaktion
that ware waiting for more than 2 minutes.
This means that you have very long transactions -- transactions that
take very long to commit.
ZEO cannot commit two transactions for the same storage at the same time.
Therefore, it sets a storage look when a transaction commit begins for
the storage.
If another transaction tries to commit to the same storage, the transaction
is blocked until the first transaction commit completes.
That's your Transaction blocked waiting for storage
When the commit is completed, then a waiting transaction is restarted.
That's your Blocked transaction restarted.
You should try to understand where the huge transactions come from.
Very often, they are caused by poor persistency design (either far
too huge objects or an immense number of tine objects or just some stupidity
(e.g. writing objects unnecessary).
I have benchmarked my Harddisk (which is at the moment an emergency system
because of a hardware failure of the main system) it has 40 MB/sec write
speed and it doesn't show high io load when we have such a hangup. I have
tried to create an object with 50 MB in the storage the ZEO server had no
problem with that. calculating this, there has to be an objekt of CD Image size
to cause the write to take more then 30 sec. But this whould
mean that the Data.fs whould grow at least a 2-3 Gig a Day (we have 5-6
such hangups a day) but it only grows arround 50-100 MB a per Day
(difference befor and after Pack) and real growth is 2-10 MB per Day.
To the number of tiny objects. I have the zeo.log on debug level. Entries
like these seam to be the objekts that are requested to be written.
2006-08-08T07:44:12 DEBUG ZEO.zrpc.Connection(S) (10.152.64.21:50210) calling
storea('\x00\x00\x00\x00\x0057\x85', '\x03gZW\x00\xa0\xa1\xcc', '(...
I don't see a lot of them bevor a hangup occur.
So there are two areas where the Problem could be located. The filesystem
of the host system and in the network between the App-Server and the Zeo.
To understand this can you tell me when a transaction is started and when
its closed. Does the Zeo server wait until all data is recieved bevor the
transaction is started or does the transaktion start when the datatransfer
from the appserver starts.
Bye
Estartu
Gerhard Schmidt| Nick : estartu IRC : Estartu |
Fischbachweg 3 || PGP Public Key
86856 Hiltenfingen | EMail: [EMAIL PROTECTED] | on request
Germany||
pgpRboq3a8VZt.pgp
Description: PGP signature
___
Zope maillist - Zope@zope.org
http://mail.zope.org/mailman/listinfo/zope
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] ZEO Problem Clients Waiting
Gerhard Schmidt wrote at 2006-8-8 08:06 +0200: ... long time commit locks ... To understand this can you tell me when a transaction is started and when its closed. Does the Zeo server wait until all data is recieved bevor the transaction is started or does the transaktion start when the datatransfer from the appserver starts. The storage lock is set when one of these function is called: vote, abortVersion, commitVersion and undo. -- Dieter ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
[Zope] ZEO Problem Clients Waiting
HI, we have quite a Problem with out Zope/Zeo system. We run at the moment Zope cluster with 10 application servers and a zeo server with Filestorage. Our Data.fs is 3.5 Gig at the moment (after Pack). I runs most of the time without problem. But sometimes things screw up 2006-08-07T14:29:19 INFO ZEO.StorageServer (97002/10.152.64.23:52518) Transaction +blocked waiting for storage. Clients waiting: 1. 2006-08-07T14:29:29 INFO ZEO.StorageServer (97002/10.152.64.18:60205) Transaction +blocked waiting for storage. Clients waiting: 2. 2006-08-07T14:29:39 INFO ZEO.StorageServer (97002/10.152.64.19:55021) Transaction +blocked waiting for storage. Clients waiting: 3. 2006-08-07T14:29:39 INFO ZEO.StorageServer (97002/10.152.64.15:62775) Transaction +blocked waiting for storage. Clients waiting: 4. 2006-08-07T14:29:41 INFO ZEO.StorageServer (97002/10.152.64.22:59407) Transaction +blocked waiting for storage. Clients waiting: 5. 2006-08-07T14:29:41 INFO ZEO.StorageServer (97002/10.152.64.20:51153) Transaction blocked waiting for storage. Clients waiting: 6. 2006-08-07T14:29:41 INFO ZEO.StorageServer (97002/10.152.64.21:52951) Transaction blocked waiting for storage. Clients waiting: 7. 2006-08-07T14:29:41 INFO ZEO.StorageServer (97002/10.152.64.14:53919) Transaction blocked waiting for storage. Clients waiting: 8. 2006-08-07T14:29:41 INFO ZEO.StorageServer (97002/10.152.64.17:54463) Transaction blocked waiting for storage. Clients waiting: 9. 2006-08-07T14:29:47 INFO ZEO.StorageServer (97002/10.152.64.16:53743) Blocked transaction restarted. Clients waiting: 8 2006-08-07T14:29:47 INFO ZEO.StorageServer (97002/10.152.64.23:52518) Blocked transaction restarted. Clients waiting: 7 2006-08-07T14:29:48 INFO ZEO.StorageServer (97002/10.152.64.23:52518) Transaction blocked waiting for storage. Clients waiting: 8. 2006-08-07T14:29:48 INFO ZEO.StorageServer (97002/10.152.64.18:60205) Blocked transaction restarted. Clients waiting: 7 2006-08-07T14:29:48 INFO ZEO.StorageServer (97002/10.152.64.19:55021) Blocked transaction restarted. Clients waiting: 6 2006-08-07T14:29:48 INFO ZEO.StorageServer (97002/10.152.64.15:62775) Blocked transaction restarted. Clients waiting: 5 2006-08-07T14:29:48 INFO ZEO.StorageServer (97002/10.152.64.22:59407) Blocked transaction restarted. Clients waiting: 4 2006-08-07T14:29:48 INFO ZEO.StorageServer (97002/10.152.64.20:51153) Blocked transaction restarted. Clients waiting: 3 2006-08-07T14:29:48 INFO ZEO.StorageServer (97002/10.152.64.21:52951) Blocked transaction restarted. Clients waiting: 2 2006-08-07T14:29:48 INFO ZEO.StorageServer (97002/10.152.64.16:53743) Transaction blocked waiting for storage. Clients waiting: 3. 2006-08-07T14:29:49 INFO ZEO.StorageServer (97002/10.152.64.14:53919) Blocked transaction restarted. Clients waiting: 2 2006-08-07T14:29:50 INFO ZEO.StorageServer (97002/10.152.64.17:54463) Blocked transaction restarted. Clients waiting: 1 2006-08-07T14:29:50 INFO ZEO.StorageServer (97002/10.152.64.23:52518) Blocked transaction restarted. This one was a very quick one only 30 seconds. I have Blocked Transaktion that ware waiting for more than 2 minutes. When there a 9 Clients waiting our complete Zopesite comes to a halt until the transaction is restarted. As im not realy deep in the ZODB/ZEO subject. Can anybody pinpoint me where to look for the Problem. Bye Estartu Gerhard Schmidt| Nick : estartu IRC : Estartu | Fischbachweg 3 || PGP Public Key 86856 Hiltenfingen | EMail: [EMAIL PROTECTED] | on request Germany|| pgpngdBFhioow.pgp Description: PGP signature ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] ZEO Problem Clients Waiting
Gerhard Schmidt wrote at 2006-8-7 15:54 +0200: ... 2006-08-07T14:29:19 INFO ZEO.StorageServer (97002/10.152.64.23:52518) Transaction +blocked waiting for storage. Clients waiting: 1. ... 2006-08-07T14:29:50 INFO ZEO.StorageServer (97002/10.152.64.17:54463) Blocked transaction restarted. Clients waiting: 1 2006-08-07T14:29:50 INFO ZEO.StorageServer (97002/10.152.64.23:52518) Blocked transaction restarted. This one was a very quick one only 30 seconds. I have Blocked Transaktion that ware waiting for more than 2 minutes. This means that you have very long transactions -- transactions that take very long to commit. ZEO cannot commit two transactions for the same storage at the same time. Therefore, it sets a storage look when a transaction commit begins for the storage. If another transaction tries to commit to the same storage, the transaction is blocked until the first transaction commit completes. That's your Transaction blocked waiting for storage When the commit is completed, then a waiting transaction is restarted. That's your Blocked transaction restarted. You should try to understand where the huge transactions come from. Very often, they are caused by poor persistency design (either far too huge objects or an immense number of tine objects or just some stupidity (e.g. writing objects unnecessary). -- Dieter ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Zeo as a Zeo Client
On Tue, May 02, 2006 at 10:24:29PM +0200, Dieter Maurer wrote: Gerhard Schmidt wrote at 2006-5-2 09:52 +0200: ... I try to reduce the load of the line between the backup Computing Center and the Mainsite by having a zeo server as Proxy between the zope server at the backup site an the ZEO at the main site. You will gain nothing -- as ZEO does not implement a cache but forwards any request immediately to the storage. Its only task is to synchronize concurrent access to a single storage -- nothing else. For my primary goal this whould do perfectly. Primarily I whan't the zeo at backup site just to foward the request to the main site. All I want is the i have only one place to change the config in case of a failure at the main site. Nothing more. Everything else whould be nice to have. But still the even just forwarding does not work right now. Bye Estartu Gerhard Schmidt| Nick : estartu IRC : Estartu | Fischbachweg 3 || PGP Public Key 86856 Hiltenfingen | EMail: [EMAIL PROTECTED] | on request Germany|| ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Zeo as a Zeo Client
Jens Vagelpohl wrote: On 2 May 2006, at 08:52, Gerhard Schmidt wrote: A configuration like that is described in the Zope Book on page 230. I'd like to see that. The proxying you describe is simply not possible, period. He's decribing the dead tree version of the Zope Book. Either Amos or Mike decided to put in some vapourware about how ZEO worked. It is there, I have a copy and can show you if you like... Chris -- Simplistix - Content Management, Zope Python Consulting - http://www.simplistix.co.uk ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Zeo as a Zeo Client
Gerhard Schmidt wrote at 2006-5-3 08:20 +0200: On Tue, May 02, 2006 at 10:24:29PM +0200, Dieter Maurer wrote: Gerhard Schmidt wrote at 2006-5-2 09:52 +0200: ... I try to reduce the load of the line between the backup Computing Center and the Mainsite by having a zeo server as Proxy between the zope server at the backup site an the ZEO at the main site. You will gain nothing -- as ZEO does not implement a cache but forwards any request immediately to the storage. Its only task is to synchronize concurrent access to a single storage -- nothing else. For my primary goal this whould do perfectly. Primarily I whan't the zeo at backup site just to foward the request to the main site. All I want is the i have only one place to change the config in case of a failure at the main site. Nothing more. Everything else whould be nice to have. But still the even just forwarding does not work right now. Why would you want to use a ZEO server just for this relaying? Would it not be better to use a DNS name which you could remap at a central place (the DNS configuration) in case of problems? Or use a TCP forwarder. A ZEO server is definitely not appropriate just to relay requests. -- Dieter ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
[Zope] Zeo as a Zeo Client
Hallo,
I am trying to install a setup like shown below
ZEO_M
| -- ZEO_B
| | -- ZOPE
| | -- ZOPE
| | -- ZOPE
| ...
| -- ZOPE
| -- ZOPE
...
The ZEO_B should be a zeoclient of the ZEO_M (MainZEO). The reason for
this setup ist that the ZEO_B is located in a backup computing center on
the other side of our campus.
When I configured the ZEO_B as a zeoclient of the ZEO_M. I comes up
without problem. I've connected the first client, the connection is
establisched but the server never come up.
Here is the log from ZEO_B level debug
2006-05-02T08:21:45 BLATHER ZEO.zrpc (65280) connect from ('10.152.64.1',
54853): ManagedServerConnection ('10.152.64.1', 54853)
--
2006-05-02T08:21:45 INFO ZEO.zrpc.Connection(S) (10.152.64.1:54853) received
handshake 'Z303'
--
2006-05-02T08:21:45 DEBUG ZEO.zrpc.Connection(S) (10.152.64.1:54853) calling
getAuthProtocol()
--
2006-05-02T08:21:45 DEBUG ZEO.zrpc.Connection(S) (10.152.64.1:54853)
getAuthProtocol returns None
--
2006-05-02T08:21:45 DEBUG ZEO.zrpc.Connection(S) (10.152.64.1:54853) calling
register('main', False)
--
2006-05-02T08:21:45 DEBUG ZEO.zrpc.Connection(S) (10.152.64.1:54853) register
returns None
--
2006-05-02T08:21:45 DEBUG ZEO.zrpc.Connection(S) (10.152.64.1:54853) calling
get_info()
--
2006-05-02T08:21:45 DEBUG ZEO.zrpc.Connection(S) (10.152.64.1:54853) get_info
returns {'supportsVersions': 1, 'name': 'main (connected)', 'length'...
--
2006-05-02T08:21:45 DEBUG ZEO.zrpc.Connection(S) (10.152.64.1:54853) calling
endZeoVerify()
--
2006-05-02T08:21:45 DEBUG ZEO.zrpc.Connection(S) (10.152.64.1:54853) calling
loadEx('\x00\x00\x00\x00\x00\x00\x00\x00', '')
here is the log for the zope server on ZEO_B
2006-05-02 08:21:45 INFO ZEO.ClientStorage (65292) ClientStorage (pid=65292)
created RW/normal for storage: 'main'
2006-05-02 08:21:45 INFO ZEO.cache reusing persistent cache file
'/data/zope/console28/var/frontend-main.zec'
2006-05-02 08:21:45 INFO ZEO.ClientStorage (65292) Testing connection
ManagedClientConnection ('10.152.64.1', 8110)
2006-05-02 08:21:45 INFO ZEO.zrpc.Connection(C) (10.152.64.1:8110) received
handshake 'Z303'
2006-05-02 08:21:45 INFO ZEO.ClientStorage (65292) Server authentication
protocol None
2006-05-02 08:21:45 INFO ZEO.ClientStorage (65292) Connected to storage:
('10.152.64.1', 8110)
2006-05-02 08:21:45 INFO ZEO.ClientStorage (65292) Verifying cache
2006-05-02 08:21:45 INFO ZEO.ClientStorage (65292) Waiting for cache
verification to finish
2006-05-02 08:21:45 INFO ZEO.ClientStorage (65292) Waiting for cache
verification to finish
2006-05-02 08:21:45 INFO ZEO.ClientStorage (65292) endVerify finishing
2006-05-02 08:21:45 INFO ZEO.ClientStorage (65292) endVerify finished
^C2006-05-02 08:22:09 INFO ZEO.ClientStorage (65292) Disconnected from storage:
('10.152.64.1', 8110)
Traceback (most recent call last):
File /usr/local/www/Zope28/lib/python/Zope2/Startup/run.py, line 56, in ?
run()
File /usr/local/www/Zope28/lib/python/Zope2/Startup/run.py, line 21, in run
starter.prepare()
File /usr/local/www/Zope28/lib/python/Zope2/Startup/__init__.py, line 98,
in prepare
self.startZope()
File /usr/local/www/Zope28/lib/python/Zope2/Startup/__init__.py, line 257,
in startZope
Zope2.startup()
File /usr/local/www/Zope28/lib/python/Zope2/__init__.py, line 47, in startup
_startup()
File /usr/local/www/Zope28/lib/python/Zope2/App/startup.py, line 57, in
startup
DB = configuration.dbtab.getDatabase('/', is_root=1)
File /usr/local/www/Zope28/lib/python/DBTab/DBTab.py, line 96, in
getDatabase
db = self._createDatabase(name, is_root)
File /usr/local/www/Zope28/lib/python/DBTab/DBTab.py, line 113, in
_createDatabase
db = factory.open()
File /usr/local/www/Zope28/lib/python/Zope2/Startup/datatypes.py, line 163,
in open
DB = self.createDB()
File /usr/local/www/Zope28/lib/python/Zope2/Startup/datatypes.py, line 160,
in createDB
return ZODBDatabase.open(self)
File /usr/local/www/Zope28/lib/python/ZODB/config.py, line 103, in open
version_cache_size=section.version_cache_size)
File /usr/local/www/Zope28/lib/python/ZODB/DB.py, line 239, in __init__
storage.load(z64,'')
File /usr/local/www/Zope28/lib/python/ZEO/ClientStorage.py, line 746, in
load
return self.loadEx(oid, version)[:2]
File /usr/local/www/Zope28/lib/python/ZEO/ClientStorage.py, line 769, in
loadEx
data, tid, ver = self._server.loadEx(oid, version)
File /usr/local/www/Zope28/lib/python/ZEO/ServerStub.py, line 192, in loadEx
return self.rpc.call(loadEx, oid, version)
File /usr/local/www/Zope28/lib/python/ZEO/zrpc/connection.py, line 531, in
call
r_flags, r_args = self.wait(msgid)
File /usr/local/www/Zope28/lib/python/ZEO/zrpc/connection.py, line 638, in
wait
asyncore.poll(delay, self._singleton)
File /usr/local/lib/python2.4/asyncore.py, line 122, in poll
r, w, e = select.select(r, w, e, timeout)
Re: [Zope] Zeo as a Zeo Client
--On 2. Mai 2006 08:31:17 +0200 Gerhard Schmidt [EMAIL PROTECTED] wrote: Hallo, I am trying to install a setup like shown below ZEO_M | -- ZEO_B | | -- ZOPE | | -- ZOPE | | -- ZOPE | ... | -- ZOPE | -- ZOPE ... You are trying to create a cascade of multiple ZEO Clients? That looks very odd. Usually a ZEO client talks directly to a ZEO server. -aj -- ZOPYX Ltd. Co. KG - Charlottenstr. 37/1 - 72070 Tübingen - Germany Web: www.zopyx.com - Email: [EMAIL PROTECTED] - Phone +49 - 7071 - 793376 E-Publishing, Python, Zope Plone development, Consulting pgpNFM1rW1QOH.pgp Description: PGP signature ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Zeo as a Zeo Client
On Tue, May 02, 2006 at 08:51:12AM +0200, Andreas Jung wrote: --On 2. Mai 2006 08:31:17 +0200 Gerhard Schmidt [EMAIL PROTECTED] wrote: Hallo, I am trying to install a setup like shown below ZEO_M | -- ZEO_B | | -- ZOPE | | -- ZOPE | | -- ZOPE | ... | -- ZOPE | -- ZOPE ... You are trying to create a cascade of multiple ZEO Clients? That looks very odd. Usually a ZEO client talks directly to a ZEO server. I try to reduce the load of the line between the backup Computing Center and the Mainsite by having a zeo server as Proxy between the zope server at the backup site an the ZEO at the main site. Secound part is that the zeo at the backupsite can easily reconfigured in a normal ZEO when the mainsite is offline. So I don't have to reconfigure all zeoclients at the backupsite. The Data.fs is copied every hour to the backup site so that a have an up to one our backup of the data.fs in case of a desaster at the main site. A configuration like that is described in the Zope Book on page 230. Besides I have found that with a growing number of zeo clients the Zeo server gets slower but neither the CPU nor the Harddisk IO is at the limit. We have a load of 0.2 to 0.3 and disk IO arrond 2-3 MB/sec. We have 12 zeoclients at the moment and 12 more are planed for the backup site. Regards Estartu Gerhard Schmidt| Nick : estartu IRC : Estartu | Fischbachweg 3 || PGP Public Key 86856 Hiltenfingen | EMail: [EMAIL PROTECTED] | on request Germany|| ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Zeo as a Zeo Client
--On 2. Mai 2006 09:52:57 +0200 Gerhard Schmidt [EMAIL PROTECTED] wrote: A configuration like that is described in the Zope Book on page 230. I can not see anything like that in the Zope Book 2.7 edition. As said a ZEO client talks to a *ZEO server* and *not* to another *ZEO client*. Anything else makes no sense. When you're look for HA solution then checkout ZRS (see on zope.com). -aj -- ZOPYX Ltd. Co. KG - Charlottenstr. 37/1 - 72070 Tübingen - Germany Web: www.zopyx.com - Email: [EMAIL PROTECTED] - Phone +49 - 7071 - 793376 E-Publishing, Python, Zope Plone development, Consulting pgpPg5u16EitE.pgp Description: PGP signature ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Zeo as a Zeo Client
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 2 May 2006, at 08:52, Gerhard Schmidt wrote: A configuration like that is described in the Zope Book on page 230. I'd like to see that. The proxying you describe is simply not possible, period. As Andreas mentioned, the (commercial) ZRS product from Zope Corp will give the the hot backup/hot standby you seem to look for. All backup ZEO servers can be accessed by clients, but read-only. jens -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.1 (Darwin) iD8DBQFEVx8IRAx5nvEhZLIRAkqnAJ973dfiJnM68Pw/Cym1pQvG3h7ddACfT6Oj ZEkv3PzqNUlZ2aIrGTSz2fk= =5/6J -END PGP SIGNATURE- ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
[Zope] ZEO ClientDisconnected error
Hi there,
One of my clients is using Plone on Solaris 9. We have set up one ZEO
client on a Solaris machine and a ZEO server on another Solaris machine
(for testing purposes only at this stage).
Performance is great but every morning, the first hit to the Plone site
returns a ClientDisconnected error (see full error log below). It looks
like the socket on the server gets corrupted or something.
When the error occurs, the server is non-responsive for a while
(anywhere between 10 sec and 2 minutes) and then eventually the server
responds.
Has anybody seen that before? Is it Zope related or should I look in
other directions?
Researching Zope on Solaris, I read somewhere (can't remember where now)
that Solaris doesn't like long-running processes too much (well, the
Zope process, for instance). Could it be the culprit here?
Any advise or pointer welcome.
Cyrille
(in the logs below, appserv05 is the ZEO client, pehi is the ZEO server)
From event.log:
=
ERROR ZEO.zrpc.Connection(C) (pehi.myclient.govt.nz:8100) Error caught
in asyncore raise socket.error, why error: (9, 'Bad file descriptor')
2006-04-28T09:09:52
=
From the error log:
=
Time
2006-04-28 09:09
User Name
admin (admin)
Request URL
http://plone.appserv05.myclient.govt.nz/front-page/document_view
Exception Type
ClientDisconnected
Exception Value
Traceback (innermost last):
•Module ZPublisher.Publish, line 114, in publish
•Module ZPublisher.mapply, line 88, in mapply
•Module ZPublisher.Publish, line 40, in call_object
•Module Shared.DC.Scripts.Bindings, line 311, in __call__
•Module Shared.DC.Scripts.Bindings, line 348, in _bindAndExec
•Module Products.CMFCore.FSPageTemplate, line 195, in _exec
•Module Products.CMFCore.FSPageTemplate, line 134, in pt_render
•Module Products.PageTemplates.PageTemplate, line 104, in pt_render
FSPageTemplate at /myclient/Plone/document_view used for
/myclient/Plone/front-page
•Module TAL.TALInterpreter, line 206, in __call__
•Module TAL.TALInterpreter, line 250, in interpret
•Module TAL.TALInterpreter, line 711, in do_useMacro
•Module TAL.TALInterpreter, line 250, in interpret
•Module TAL.TALInterpreter, line 426, in do_optTag_tal
•Module TAL.TALInterpreter, line 411, in do_optTag
•Module TAL.TALInterpreter, line 406, in no_tag
•Module TAL.TALInterpreter, line 250, in interpret
•Module TAL.TALInterpreter, line 711, in do_useMacro
•Module TAL.TALInterpreter, line 250, in interpret
•Module TAL.TALInterpreter, line 481, in do_setGlobal_tal
•Module Products.PageTemplates.TALES, line 221, in evaluate
URL: file:CMFPlone/skins/plone_templates/global_defines.pt
Line 3, Column 0
Expression: PythonExpr language or here.Language() or default_language
Names:
•{'container': PloneSite at /myclient/Plone,
• 'context': ATDocument at /myclient/Plone/front-page,
• 'default': Products.PageTemplates.TALES.Default instance at
0xacaf58,
• 'here': ATDocument at /myclient/Plone/front-page,
• 'loop': Products.PageTemplates.TALES.SafeMapping object at
0x22027d8,
• 'modules':
Products.PageTemplates.ZRPythonExpr._SecureModuleImporter instance at
0xabd5d0,
• 'nothing': None,
• 'options': {'args': ()},
• 'repeat': Products.PageTemplates.TALES.SafeMapping object at
0x22027d8,
• 'request': HTTPRequest,
URL=http://plone.appserv05.myclient.govt.nz/front-page/document_view,
• 'root': Application at ,
• 'template': FSPageTemplate at /myclient/Plone/document_view used
for /myclient/Plone/front-page,
• 'traverse_subpath': [],
'user': admin}
•Module Products.PageTemplates.ZRPythonExpr, line 47, in __call__
__traceback_info__: language or here.Language() or default_language
•Module Python expression language or here.Language() or
default_language, line 1, in expression
•Module Products.Archetypes.ClassGen, line 58, in generatedAccessor
•Module Products.Archetypes.Field, line 768, in get
•Module Products.Archetypes.Field, line 637, in get
__traceback_info__: ('language', ATDocument at
/myclient/Plone/front-page, {'field': Field language(string:rw),
'schema': Products.Archetypes.Schema.Schema object at 0x171a330})
•Module Products.Archetypes.Storage, line 175, in get
•Module UserDict, line 19, in __getitem__
•Module ZODB.Connection, line 704, in setstate
•Module ZODB.Connection, line 760, in _setstate
•Module ZODB.serialize, line 495, in setGhostState
•Module ZODB.serialize, line 488, in getState
•Module ZODB.serialize, line 436, in _persistent_load
•Module ZODB.Connection, line 207, in get
•Module ZEO.ClientStorage, line 746, in load
•Module ZEO.ClientStorage, line 769, in loadEx
•Module ZEO.ClientStorage, line
Re: [Zope] Zope=ZEO connection
Dennis Allison wrote at 2006-2-7 18:18 -0800: ... What sort of Zope failure can cause this sort of behavior? What's the best approach to get more information to localize the failure. A crash presented to Zope as a fatal signal (usually SIGSEGV or SIGBUS). Reconfigure your Linux account running Zope to write core files (ulimit -Sc 10). With a bit of luck you will get a core file written in case of such crashes. Look into the core with GDB. Note, that the value 10 might not be big enough to get a complete core file. GDB may have problems to really understand incomplete ones. -- Dieter ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Zope=ZEO connection
Dennis Allison wrote: 2006-02-06T14:07:20 INFO ZPublisher.Conflict ConflictError at some path: database conflict error (oid 0x086e, class BTrees._OOBTree.OOBTree, serial this txn started with 0x03633ca95f75e900 2006-02-06 22:01:22.373575, serial currently committed 0x03633caf59114244 2006-02-06 22:07:20.875176) (463 conflicts (0 unresolved) since startup at Mon Feb 6 08:21:26 2006) Well, at least the new conflict error logging stuff works :-) -- 2006-02-06T14:09:09 INFO ZServer HTTP server started at Mon Feb 6 14:09:09 2006 Hostname: x-harper Port: 8081 which also triggers a connection drop and restarBt of ZEO. Odd... not seen this at all, what OS you running on? cheers, Chris -- Simplistix - Content Management, Zope Python Consulting - http://www.simplistix.co.uk ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
[Zope] Zope=ZEO connection
Zope 2.9.0 Python 2.4.2 Amd dual Opteron Linux I am seeing occasional (several per day) Zope crashes under load. The event.log immediately before and after the crash looks like: 2006-02-06T14:07:20 INFO ZPublisher.Conflict ConflictError at some path: database conflict error (oid 0x086e, class BTrees._OOBTree.OOBTree, serial this txn started with 0x03633ca95f75e900 2006-02-06 22:01:22.373575, serial currently committed 0x03633caf59114244 2006-02-06 22:07:20.875176) (463 conflicts (0 unresolved) since startup at Mon Feb 6 08:21:26 2006) -- 2006-02-06T14:09:09 INFO ZServer HTTP server started at Mon Feb 6 14:09:09 2006 Hostname: x-harper Port: 8081 which also triggers a connection drop and restarBt of ZEO. The failure happens silently even with eventlog level ALL logfile path $INSTANCE/log/event.log level info /logfile /eventlog When this happens zope is running under zopectl and zeo isi running under zeoctl. What sort of Zope failure can cause this sort of behavior? What's the best approach to get more information to localize the failure. ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Zope=ZEO connection
On Tue, Feb 07, 2006 at 06:18:49PM -0800, Dennis Allison wrote: Zope 2.9.0 Python 2.4.2 Amd dual Opteron Linux I am seeing occasional (several per day) Zope crashes under load. The event.log immediately before and after the crash looks like: 2006-02-06T14:07:20 INFO ZPublisher.Conflict ConflictError at some path: database conflict error (oid 0x086e, class BTrees._OOBTree.OOBTree, serial this txn started with 0x03633ca95f75e900 2006-02-06 22:01:22.373575, serial currently committed 0x03633caf59114244 2006-02-06 22:07:20.875176) (463 conflicts (0 unresolved) since startup at Mon Feb 6 08:21:26 2006) -- 2006-02-06T14:09:09 INFO ZServer HTTP server started at Mon Feb 6 14:09:09 2006 Hostname: x-harper Port: 8081 which also triggers a connection drop and restarBt of ZEO. The failure happens silently even with eventlog level ALL logfile path $INSTANCE/log/event.log level info ^^ fwiw, you're still only logging at INFO. What sort of Zope failure can cause this sort of behavior? What's the best approach to get more information to localize the failure. I really don't know. But I'd have a look at the trace log, if you've got that configured (I routinely do). Maybe you could spot some patterns right before the crashes? *shrug* -- Paul Winkler http://www.slinkp.com ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] Zope=ZEO connection
Arh... I forgot that internal logging levels override. Still, you'd think something that triggers a crash would have a level higher than info. Careful examination of the trace log shows nothing. I'll fix the log level see if that turns anything up. On Tue, 7 Feb 2006, Paul Winkler wrote: On Tue, Feb 07, 2006 at 06:18:49PM -0800, Dennis Allison wrote: Zope 2.9.0 Python 2.4.2 Amd dual Opteron Linux I am seeing occasional (several per day) Zope crashes under load. The event.log immediately before and after the crash looks like: 2006-02-06T14:07:20 INFO ZPublisher.Conflict ConflictError at some path: database conflict error (oid 0x086e, class BTrees._OOBTree.OOBTree, serial this txn started with 0x03633ca95f75e900 2006-02-06 22:01:22.373575, serial currently committed 0x03633caf59114244 2006-02-06 22:07:20.875176) (463 conflicts (0 unresolved) since startup at Mon Feb 6 08:21:26 2006) -- 2006-02-06T14:09:09 INFO ZServer HTTP server started at Mon Feb 6 14:09:09 2006 Hostname: x-harper Port: 8081 which also triggers a connection drop and restarBt of ZEO. The failure happens silently even with eventlog level ALL logfile path $INSTANCE/log/event.log level info ^^ fwiw, you're still only logging at INFO. What sort of Zope failure can cause this sort of behavior? What's the best approach to get more information to localize the failure. I really don't know. But I'd have a look at the trace log, if you've got that configured (I routinely do). Maybe you could spot some patterns right before the crashes? *shrug* -- ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
[Zope] ZEO disconnects, Zope auto restarts (via zopectl)
Zope 2.9.0 We are seeing spontaneous restarts of Zope with no indication in any of the standard Zope logs. Looking at the ZEO log indicates that the restarts of Zope are due to a lost connection between Zope ZEO but with no other information. The logging level is set at the distribution default (INFO). The restarts are a huge problem because session variables are not persistent and so all of the user state they contain is lost on restart. In our statful implementation, this is a major problem. I want to adjust the configuration so that the Zope/ZEO connection is stable. In our configuration, Zope and ZEO are linked via localhost on a distinguished port. I've Googled about looking for some infomation about tuning the ZEO/Zope interface, but have found little real information. Some additional log detail would be helpful. We are running a fairly vanilla setup, excerpted below: zope.conf # ZEO client storage: # zodb_db main mount-point / # ZODB cache, in number of objects cache-size 5000 zeoclient server localhost:8301 storage 1 var $INSTANCE/var # ZEO client cache, in bytes cache-size 20MB # Uncomment to have a persistent disk cache client group1-zeo /zeoclient /zodb_db zeo.conf zeo address localhost:8301 read-only false invalidation-queue-size 100 pid-filename $INSTANCE/var/ZEO.pid # monitor-address PORT # transaction-timeout SECONDS /zeo runner program $INSTANCE/bin/runzeo socket-name $INSTANCE/etc/zeo.zdsock daemon true forever false backoff-limit 10 exit-codes 0, 2 directory $INSTANCE default-to-interactive true # user zope python /usr/bin/python2.4 zdrun /usr/local/src/zope/Zope2.9/lib64/python/zdaemon/zdrun.py # This logfile should match the one in the zeo.conf file. # It is used by zdctl's logtail command, zdrun/zdctl doesn't write it. logfile $INSTANCE/log/zeo.log /runner It's not clear what changes will lead to a more stable connection because it is not clear what's triggering the problem. Any advice would be appreciated. Presumably the shotgun approach would work -- increase the cache sizes, lengthen the invalidation-queue-size, and increase the backoff-limit but it would be nice to have some guidance. 5~ ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] ZEO disconnects, Zope auto restarts (via zopectl)
On Fri, Feb 03, 2006 at 01:00:45AM -0800, Dennis Allison wrote: Zope 2.9.0 We are seeing spontaneous restarts of Zope with no indication in any of the standard Zope logs. Looking at the ZEO log indicates that the restarts of Zope are due to a lost connection between Zope ZEO but with no other information. The logging level is set at the distribution default (INFO). Are you *sure* that is the cause, rather than the effect? If zope restarts for any reason, I'd expect the zeo log to show a disconnect and reconnect as a result. Check the clocks on your zope and zeo boxes and make sure the timing of events in your logs is really what you think it is. (Systems that aren't running ntpd are the bane of my existence...) Wild guess: Any chance your Zope process is running out of memory? I've had that on several occasions, when some naively-written software attempts to do something huge in memory that should really use a temp file on disk. (Zope itself used to have some code like that in the FTP server, don't know if it still does.) I discovered this by looking in /var/log/messages. At least on linux, the kernel will log something there when it kills a process that consumes all available memory. We are running a fairly vanilla setup, excerpted below: zope.conf # ZEO client storage: # zodb_db main mount-point / # ZODB cache, in number of objects cache-size 5000 zeoclient server localhost:8301 storage 1 var $INSTANCE/var # ZEO client cache, in bytes cache-size 20MB Unrelated to your problem, and maybe you know this, but depending on the size of your storage, I'd consider increasing the zeo client cache size. It's a disk cache and you can safely make it huge. But if you don't see cache flipping messages in your event log, it may not matter. # Uncomment to have a persistent disk cache client group1-zeo /zeoclient /zodb_db zeo.conf zeo address localhost:8301 read-only false invalidation-queue-size 100 pid-filename $INSTANCE/var/ZEO.pid # monitor-address PORT # transaction-timeout SECONDS /zeo runner program $INSTANCE/bin/runzeo socket-name $INSTANCE/etc/zeo.zdsock daemon true forever false backoff-limit 10 exit-codes 0, 2 directory $INSTANCE default-to-interactive true # user zope python /usr/bin/python2.4 zdrun /usr/local/src/zope/Zope2.9/lib64/python/zdaemon/zdrun.py # This logfile should match the one in the zeo.conf file. # It is used by zdctl's logtail command, zdrun/zdctl doesn't write it. logfile $INSTANCE/log/zeo.log /runner It's not clear what changes will lead to a more stable connection because it is not clear what's triggering the problem. Any advice would be appreciated. Presumably the shotgun approach would work -- increase the cache sizes, lengthen the invalidation-queue-size, and increase the backoff-limit but it would be nice to have some guidance. 5~ ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev ) -- Paul Winkler http://www.slinkp.com ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] ZEO disconnects, Zope auto restarts (via zopectl)
Paul, Thanks for the assist. Comments on your comments interlinearly below. I have increased cache and other resources to see what the impact will be. On Fri, 3 Feb 2006, Paul Winkler wrote: On Fri, Feb 03, 2006 at 01:00:45AM -0800, Dennis Allison wrote: Zope 2.9.0 We are seeing spontaneous restarts of Zope with no indication in any of the standard Zope logs. Looking at the ZEO log indicates that the restarts of Zope are due to a lost connection between Zope ZEO but with no other information. The logging level is set at the distribution default (INFO). Are you *sure* that is the cause, rather than the effect? No, I am not and there's nothing in the logs which hints at why it restarted. We are running under load. The failures are silent. We do have a fairly high rate of conflict errors (which all get resolved finally!). If zope restarts for any reason, I'd expect the zeo log to show a disconnect and reconnect as a result. Check the clocks on your zope and zeo boxes and make sure the timing of events in your logs is really what you think it is. (Systems that aren't running ntpd are the bane of my existence...) Timing correlates to the second. Zope and ZEO live on the same physical box. Wild guess: Any chance your Zope process is running out of memory? I've had that on several occasions, when some naively-written software attempts to do something huge in memory that should really use a temp file on disk. (Zope itself used to have some code like that in the FTP server, don't know if it still does.) I doubt if I am hitting a limit. The box has nearly 8GB of memory most of which (6GB) is used by linux as a cache. No messages in the logs. I discovered this by looking in /var/log/messages. At least on linux, the kernel will log something there when it kills a process that consumes all available memory. We are running a fairly vanilla setup, excerpted below: [snip...] Unrelated to your problem, and maybe you know this, but depending on the size of your storage, I'd consider increasing the zeo client cache size. It's a disk cache and you can safely make it huge. But if you don't see cache flipping messages in your event log, it may not matter. Done, but I cannot report on the effect. ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] ZEO disconnects, Zope auto restarts (via zopectl)
On Fri, Feb 03, 2006 at 08:35:11AM -0800, Dennis Allison wrote: Timing correlates to the second. Zope and ZEO live on the same physical box. OK. Do you have more than one ZEO client? If not, I'd reevaluate whether you need ZEO at all. (It's great for zopectl debug on a live system, but otherwise it does nothing but add overhead if you're not using it to run multiple Zopes. But you probably knew that.) I doubt if I am hitting a limit. The box has nearly 8GB of memory most of which (6GB) is used by linux as a cache. No messages in the logs. OK. It should be pretty obvious if you were hitting a limit. I don't think it's possible on linux to run out of memory without the kernel complaining somewhere in /var/log. Unrelated to your problem, and maybe you know this, but depending on the size of your storage, I'd consider increasing the zeo client cache size. It's a disk cache and you can safely make it huge. But if you don't see cache flipping messages in your event log, it may not matter. Done, but I cannot report on the effect. Well, as I said, it's very unlikely to have any impact on your problem. If I were in your shoes the first thing I'd do is bump up the log levels on both zope and zeo to BLATHER. Adds overhead I know, but you need to find the problem somehow... it's a weird one, I've never seen zope restart for no reason. -- Paul Winkler http://www.slinkp.com ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] ZEO disconnects, Zope auto restarts (via zopectl)
On Feb 3, 2006, at 1:06 PM, Paul Winkler wrote: If I were in your shoes the first thing I'd do is bump up the log levels on both zope and zeo to BLATHER. Adds overhead I know, but you need to find the problem somehow... it's a weird one, I've never seen zope restart for no reason. This can be a symptom of a segfault if you've got zope running under a daemon manager like zopectl/zdaemon/supervisord. - C ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] ZEO client authentication
Mika, David P (Research) wrote: I see that ZEO supports a simple digest authenitcation of clients and that this is set in the zeo's config file with the keys: authentication-protocol, authentication-database and authentication-realm. Further, I can see that a client can connect using e.g. ZEO.ClientStorage with arguments for a userid password (set in the ZEO authentication database). However, how does one configure Zope (as a client to the ZEO server) to use this authentication? I see nothing in the Zope config file. Somehow Zope must be able to pass a valid userid and password in the authentication database to ZEO. The best place to ask these questions is the zodb-dev@zope.org mailing list... cheers, Chris -- Simplistix - Content Management, Zope Python Consulting - http://www.simplistix.co.uk ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
RE: [Zope] ZEO client authentication
Thanks! Works great. I found that you can generate the encrypted password with Lib/python/ZEO/zeopasswd.py I wish there was a better solution than having the cleartext password in the zope.conf file Dave -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] Behalf Of Sascha Ottolski Sent: Monday, October 03, 2005 11:40 AM To: zope@zope.org Subject: Re: [Zope] ZEO client authentication Am Montag, 3. Oktober 2005 14:44 schrieb Mika, David P (Research): I see that ZEO supports a simple digest authenitcation of clients and that this is set in the zeo's config file with the keys: authentication-protocol, authentication-database and authentication-realm. Further, I can see that a client can connect using e.g. ZEO.ClientStorage with arguments for a userid password (set in the ZEO authentication database). However, how does one configure Zope (as a client to the ZEO server) to use this authentication? I see nothing in the Zope config file. Somehow Zope must be able to pass a valid userid and password in the authentication database to ZEO. Dave Hi Dave, this might help, although it's not written by me so I'm not 100% sure that I've picked up everything: small patch to lib/python/ZODB: --- config.py(revision 37730) +++ config.py(working copy) @@ -150,7 +150,10 @@ max_disconnect_poll=self.config.max_disconnect_poll, wait=self.config.wait, read_only=self.config.read_only, -read_only_fallback=self.config.read_only_fallback) +read_only_fallback=self.config.read_only_fallback, +username=self.config.username, +password=self.config.password, +realm=self.config.realm) class BDBStorage(BaseConfig): Index: component.xml === --- component.xml(revision 37730) +++ component.xml(working copy) @@ -132,6 +132,16 @@ read_only_fallback should be true. /description /key +key name=username required=no + description +The authentication username of the server. + /description +/key +key name=password required=no + description +The authentication password of the server. + /description +/key key name=realm required=no description The authentication realm of the server. Some authentication something like this in your zeo.conf zeo section: zeo ... authentication-protocol digest authentication-database $INSTANCE/etc/auth.db authentication-realm your_realm ... /zeo obviously, a username/password in $INSTANCE/etc/auth.db (ZEO instance, that is): $ cat /mnt/zope/ZEOHome/etc/auth.db realm your_realm your_username: your_crypted_password and something like this in your zope.conf: zodb_db main mount-point / cache-size 1 zeoclient server localhost:1234 username your_username password your_cleartext_password ... ... /zeoclient /zodb_db main now, I'm not sure how to create the encrypted password, I guess zpasswd.py will help. Good luck, Sascha -- Gallileus - the power of knowledge Gallileus GmbH http://www.gallileus.info/ Pintschstraße 16 fon +49-(0)30-41 93 43 43 10249 Berlin fax +49-(0)30-41 93 43 45 Germany ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev ) ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
[Zope] ZEO client authentication
I see that ZEO supports a simple digest authenitcation of clients and that this is set in the zeo's config file with the keys: authentication-protocol, authentication-database and authentication-realm. Further, I can see that a client can connect using e.g. ZEO.ClientStorage with arguments for a userid password (set in the ZEO authentication database). However, how does one configure Zope (as a client to the ZEO server) to use this authentication? I see nothing in the Zope config file. Somehow Zope must be able to pass a valid userid and password in the authentication database to ZEO. Dave ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] ZEO client authentication
Am Montag, 3. Oktober 2005 14:44 schrieb Mika, David P (Research): I see that ZEO supports a simple digest authenitcation of clients and that this is set in the zeo's config file with the keys: authentication-protocol, authentication-database and authentication-realm. Further, I can see that a client can connect using e.g. ZEO.ClientStorage with arguments for a userid password (set in the ZEO authentication database). However, how does one configure Zope (as a client to the ZEO server) to use this authentication? I see nothing in the Zope config file. Somehow Zope must be able to pass a valid userid and password in the authentication database to ZEO. Dave Hi Dave, this might help, although it's not written by me so I'm not 100% sure that I've picked up everything: small patch to lib/python/ZODB: --- config.py(revision 37730) +++ config.py(working copy) @@ -150,7 +150,10 @@ max_disconnect_poll=self.config.max_disconnect_poll, wait=self.config.wait, read_only=self.config.read_only, -read_only_fallback=self.config.read_only_fallback) +read_only_fallback=self.config.read_only_fallback, +username=self.config.username, +password=self.config.password, +realm=self.config.realm) class BDBStorage(BaseConfig): Index: component.xml === --- component.xml(revision 37730) +++ component.xml(working copy) @@ -132,6 +132,16 @@ read_only_fallback should be true. /description /key +key name=username required=no + description +The authentication username of the server. + /description +/key +key name=password required=no + description +The authentication password of the server. + /description +/key key name=realm required=no description The authentication realm of the server. Some authentication something like this in your zeo.conf zeo section: zeo ... authentication-protocol digest authentication-database $INSTANCE/etc/auth.db authentication-realm your_realm ... /zeo obviously, a username/password in $INSTANCE/etc/auth.db (ZEO instance, that is): $ cat /mnt/zope/ZEOHome/etc/auth.db realm your_realm your_username: your_crypted_password and something like this in your zope.conf: zodb_db main mount-point / cache-size 1 zeoclient server localhost:1234 username your_username password your_cleartext_password ... ... /zeoclient /zodb_db main now, I'm not sure how to create the encrypted password, I guess zpasswd.py will help. Good luck, Sascha -- Gallileus - the power of knowledge Gallileus GmbH http://www.gallileus.info/ Pintschstraße 16 fon +49-(0)30-41 93 43 43 10249 Berlin fax +49-(0)30-41 93 43 45 Germany ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] ZEO clients eating up / filesystem
Perhaps your FS grows because ZEO write the cache files to /tmp (which is
on your root filesystem)?
-aj
--On 30. September 2005 14:06:16 +0200 Reinoud van Leeuwen
[EMAIL PROTECTED] wrote:
Hi,
We have a ZEO setup with a Plone based intranet, and for some reason it
seems that Zope is filling up space in the root filesystem. I have to
restart Zope twice a day to clear it up (restarting Zope clears up the
space). lsof does not list anything open in the root filesystem and
swapping is done in the swap partition.
Has anyone observed the same and found a solution?
System details:
- FreeBSD 4.10-RELEASE
- Pyhton 2.3.4
- Zope 2.7.3
- SOFTWARE_HOME /usr/local/zope/lib/python
- ZOPE_HOME /usr/local/zope
- INSTANCE_HOME /usr/local/WWW/intranet/zeo_client1
(2 CPU FreeBSD machines running two ZEO instances)
FilesystemSize Used Avail Capacity Mounted on
/dev/da0s1a 1032M 207M 742M22%/
/dev/da0s1e 4.1G 519M 3.3G14%/usr
/dev/da0s1g 133G 1.3G 121G 1%/usr/local
/dev/da0s1f 4.1G53M 3.7G 1%/var
procfs4.1K 4.1K 0B 100%/proc
This is the zope startup script in /usr/local/etc/rc.d
# !/bin/sh
# Start or stop zope
# $FreeBSD: ports/www/zope/files/zope.sh,v 1.4 2004/11/16 00:02:25 pav Exp
# $ PROVIDE: zope
# REQUIRE: DAEMON
# BEFORE: LOGIN
# KEYWORD: FreeBSD shutdown
#
prefix=/usr/local
# Define these zope_* variables in one of these files:
# /etc/rc.conf
# /etc/rc.conf.local
# /etc/rc.conf.d/zope
#
# DO NOT CHANGE THESE DEFAULT VALUES HERE
#
zope_enable=${zope_enable:-NO}# Enable gkrellmd
zope_instances=${zope_instances:-}# List of instancehome dirs
. /usr/local/etc/rc.subr
name=zope
rcvar=`set_rcvar`
load_rc_config $name
if checkyesno zope_enable; then
case $1 in
start)
echo Starting Zope
;;
stop)
echo Stopping Zope
;;
restart)
echo Restarting Zope
;;
*)
echo Unknown action \$1\
;;
esac
last_zope_instance=`echo $zope_instances | /usr/bin/awk {'print
$NF'}`
for instance in $zope_instances
do
if [ -r ${instance}/etc/${name}.conf -a -x
${instance}/bin/zopectl ]; then
echo -nInstance ${instance} -
${instance}/bin/zopectl $1
if [ $instance != $last_zope_instance ]; then
echo sleeping 40 seconds between
instances to prevent read conflict errors during startup
sleep 40
fi
fi
done
fi
--
__
Nothing is as subjective as reality
Reinoud van Leeuwen[EMAIL PROTECTED]
http://www.xs4all.nl/~reinoud
__
___
Zope maillist - Zope@zope.org
http://mail.zope.org/mailman/listinfo/zope
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope-dev )
pgpWHYIAYCz4Y.pgp
Description: PGP signature
___
Zope maillist - Zope@zope.org
http://mail.zope.org/mailman/listinfo/zope
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] ZEO clients eating up / filesystem
On Fri, Sep 30, 2005 at 02:13:34PM +0200, Andreas Jung wrote: Perhaps your FS grows because ZEO write the cache files to /tmp (which is on your root filesystem)? No I checked that. Nothing in /tmp (and that would have been listed by lsof) -- __ Nothing is as subjective as reality Reinoud van Leeuwen[EMAIL PROTECTED] http://www.xs4all.nl/~reinoud __ ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
[Zope] ZEO on Windows
Hi Does anyone know if the 2.0.5 windows release of Plone includes ZEO? Is there any configuration information on this? Also does Zope 2.8 for linux have ZEO as standard. Finally, if you don't run ZEO does this mean that Zope can only handle one request at a time? Thanks ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] ZEO on Windows
michael nt milne wrote at 2005-8-24 11:52 +0100: Does anyone know if the 2.0.5 windows release of Plone includes ZEO? Do you have installed it? Then see whether the Zope part contains the folder ZEO (usually in lib/python). If it does, then it contains ZEO. Also does Zope 2.8 for linux have ZEO as standard. Yes, I think so. Again: install and check whether the ZEO part is there. Finally, if you don't run ZEO does this mean that Zope can only handle one request at a time? No. It only does mean that a single process can access your ZODB data. A standard Zope process has 4 worker threads that can execute requests in parallel. Note, however, that even when you run on a multi-processor maschine, the various Python threads may not be able to truely run in parallel (a single process allows only a single thread to execute Python code; other threads may wait on something external or execute non Python (e.g. C) code, but they cannot execute Python code truely in parallel). -- Dieter ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] ZEO on Windows
[michael nt milne] ... Also does Zope 2.8 for linux have ZEO as standard. All versions of Zope at and after 2.7 include ZEO, and regardless of platform (Linux, Windows, Solaris, ..., doesn't matter). ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
Re: [Zope] ZEO on Windows
michael nt milne wrote: Hi Does anyone know if the 2.0.5 windows release of Plone includes ZEO? Ask on a Plone list. Is there any configuration information on this? Dunno what you mean. Also does Zope 2.8 for linux have ZEO as standard. Yes. Finally, if you don't run ZEO does this mean that Zope can only handle one request at a time? No. Chris -- Simplistix - Content Management, Zope Python Consulting - http://www.simplistix.co.uk ___ Zope maillist - Zope@zope.org http://mail.zope.org/mailman/listinfo/zope ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope-dev )
