Re: ZEO caching,was Re: [Zope-dev] more Zope2.6 fun: ZEO client death.
OK... more info... here's my setup. 1 ZEO server, 3 clients (client 1, client 2, client 3). client 1 is on the same physical machine as the server. client 2 is on a different machine, but close by on a high-bandwidth network. client 3 is on a third machine on the same network. If I make changes on client 1: they are not seen on client 2 OR client 3 until I flush the database cache on that client. If I make changes on client 2: they are immediately seen on client 1. client 3 does not see them until i flush its database cache. If I make changes on client 3: they are immediately seen on client 1. client 2 does not see them until i flush its database cache. I noticed that the system time on each machine was a bit off, but that doesn't seem to be the culprit; they're very close now, and it makes no difference which one is ahead of the other: I get the same behavior described above regardless of whether clients 2 and 3 are slightly ahead or slightly behind client 1. Next, will investigate ZEO logs as Shane suggested... -- Paul Winkler http://www.slinkp.com Look! Up in the sky! It's NEGA SOCK IN PARIS! (random hero from isometric.spaceninja.com) ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: ZEO caching, was Re: [Zope-dev] more Zope2.6 fun: ZEO client death.
On Thu, Jul 31, 2003 at 12:13:40PM -0400, Shane Hathaway wrote: > >This is clearly problematic. At what point is client 2 supposed to > >see the new object?? > > Immediately. You are experiencing a bug. ;-) OK... is this a known bug? Is it a bug in ZEO or Zope or what? -- Paul Winkler http://www.slinkp.com Look! Up in the sky! It's MAFIOSO LETTUCE-WAVER ! (random hero from isometric.spaceninja.com) ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
ZEO caching, was Re: [Zope-dev] more Zope2.6 fun: ZEO client death.
On Thu, Jul 31, 2003 at 10:06:08AM -0400, Shane Hathaway wrote: > Jim and PythonLabs put a bunch of work into optimizing ZEO cache > validation earlier this year, and now it's quite speedy. Which reminds me... Is it expected that ZEO can take quite a while to show a new object on all clients? With zope 2.6.2b3, ZEO 2.0.2, I can reliably get this behavior: 1) browser 1 looking at "foofolder" on client 1 adds a new object "/foofolder/bar". 2) browser 2 on client 2 visits "/foofolder/bar". Not found. 3) browser 2 on client 2 flushes the browser cache, quits / restarts the browser, etc. "/foofolder/bar" again. Still not found. 3.5) Repeat step 3 intermittently for an hour (!). Not found. 4) client 2 is restarted. Browser 2 refreshes and immediately gets the expected object "bar". This is clearly problematic. At what point is client 2 supposed to see the new object?? -- Paul Winkler http://www.slinkp.com Look! Up in the sky! It's SADIST ITE! (random hero from isometric.spaceninja.com) ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] more Zope2.6 fun: ZEO client death.
Andrew Sydelko wrote: It's faster especially compared to ZEO 1.x, but also compared to earlier versions of ZEO 2.x/ZEO from ZODB3.1.x. At one point, changes were made to the initial cache invalidation check that caused ZEO startup time to become significantly faster. Before this change the ZEO client would take more than a minute to start (with a cache_size of 200MB). We've not tried ZEO from ZODB3.2.x yet. Oh, right. I was concerned for a moment that a minor change in ZEO 3.2 had slowed it down somehow. But you weren't talking about that. Whew. Jim and PythonLabs put a bunch of work into optimizing ZEO cache validation earlier this year, and now it's quite speedy. Shane ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] more Zope2.6 fun: ZEO client death.
On Thu, 31 Jul 2003 8:49am, Shane Hathaway wrote: Andrew Sydelko wrote: I've seen no problems with imports. And if anything, the cache implementation in the ZODB3.1.2 version of ZEO made a remarkable performance difference. ... as compared with earlier versions or later versions? I hope it's getting faster, not slower. It's faster especially compared to ZEO 1.x, but also compared to earlier versions of ZEO 2.x/ZEO from ZODB3.1.x. At one point, changes were made to the initial cache invalidation check that caused ZEO startup time to become significantly faster. Before this change the ZEO client would take more than a minute to start (with a cache_size of 200MB). We've not tried ZEO from ZODB3.2.x yet. --andy. ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] more Zope2.6 fun: ZEO client death.
Andrew Sydelko wrote: I've seen no problems with imports. And if anything, the cache implementation in the ZODB3.1.2 version of ZEO made a remarkable performance difference. ... as compared with earlier versions or later versions? I hope it's getting faster, not slower. :-) Shane ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] more Zope2.6 fun: ZEO client death.
On Thu, 31 Jul 2003 12:57am, Anthony Baxter wrote: Anthony Baxter wrote Updating it by hand to ZEO2 hasn't made all good and happy, though - the ZEO/start.py includes import ThreadedAsync.LoopCallback at the top of the file, but the sys.path magic that makes this available is inside the main() function. Moving the import to the line after the sys.path magic fixes this. I'm now getting a failed import for ZODB.utils.oid_repr, which suggests that I'm going to need to update the ZODB installation here from the one that's in 2.6.2. :-/ Ok, this was easy - Zope2.6 requires the ZODB3-3_1-branch ZEO code. The previous ThreadedAsync import fix still needs to be applied, though... Why? I'm using Zope 2.6.2b4 and moving the ZEO directory out of ZODB3.1.2 and into Zope/lib/python/ and everything works fine. I've seen no problems with imports. And if anything, the cache implementation in the ZODB3.1.2 version of ZEO made a remarkable performance difference. --andy. ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] more Zope2.6 fun: ZEO client death.
>>> Anthony Baxter wrote > Updating it by hand to ZEO2 hasn't made all good and happy, though - > the ZEO/start.py includes > import ThreadedAsync.LoopCallback > at the top of the file, but the sys.path magic that makes this > available is inside the main() function. Moving the import to > the line after the sys.path magic fixes this. > > I'm now getting a failed import for ZODB.utils.oid_repr, which > suggests that I'm going to need to update the ZODB installation > here from the one that's in 2.6.2. :-/ Ok, this was easy - Zope2.6 requires the ZODB3-3_1-branch ZEO code. The previous ThreadedAsync import fix still needs to be applied, though... Anthony ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] more Zope2.6 fun: ZEO client death.
Jim and Andrew hit the nail on the head - excessive cleverness with vendor branches here meant that the ZEO directory had ZEO1 installed. Updating it by hand to ZEO2 hasn't made all good and happy, though - the ZEO/start.py includes import ThreadedAsync.LoopCallback at the top of the file, but the sys.path magic that makes this available is inside the main() function. Moving the import to the line after the sys.path magic fixes this. I'm now getting a failed import for ZODB.utils.oid_repr, which suggests that I'm going to need to update the ZODB installation here from the one that's in 2.6.2. :-/ -- Anthony Baxter <[EMAIL PROTECTED]> It's never too late to have a happy childhood. ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] more Zope2.6 fun: ZEO client death.
On Wed, 30 Jul 2003 17:21:51 +1000 Anthony Baxter <[EMAIL PROTECTED]> wrote: > So I see ZEO clients here falling over all over the place under current > 2.6 with: > > 2003-07-30T07:01:04 ERROR(200) ZEO uncaptured python exception, > closing channel > . . [Explanation of problem snipped] . > > This, to me, seems a showstopper for 2.6.2. We've been running a ZEO client / ZEO server setup using Zope 2.6.2b2,3 and 4 and not encountered this problem. Which ZEO version are you using? --andy. ___ Zope-Dev maillist - [EMAIL PROTECTED] http://mail.zope.org/mailman/listinfo/zope-dev ** No cross posts or HTML encoding! ** (Related lists - http://mail.zope.org/mailman/listinfo/zope-announce http://mail.zope.org/mailman/listinfo/zope )
Re: [Zope-dev] more Zope2.6 fun: ZEO client death.
>>> Anthony Baxter wrote
> So I see ZEO clients here falling over all over the place under current
> 2.6 with:
>
> 2003-07-30T07:01:04 ERROR(200) ZEO uncaptured python exception,
> closing channel
>
> (exceptions.AttributeError:keys
> [/export/01/zope/dev_ekit/ZServer/medusa/asyncore.py|poll|94]
> [/export/01/zope/dev_ekit/ZServer/medusa/asyncore.py|handle_read_event|395]
> [/export/01/zope/dev_ekit/lib/python/ZEO/smac.py|handle_read|172]
> [/export/01/zope/dev_ekit/lib/python/ZEO/zrpc.py|message_input|282]
> [/export/01/zope/dev_ekit/lib/python/ZEO/ClientStorage.py|out_of_band_hook|191]
> [/export/01/zope/dev_ekit/lib/python/ZEO/Invalidator.py|end|126]
> [/export/01/zope/dev_ekit/lib/python/ZODB/DB.py|invalidate|318])
For what it's worth, the following patch "makes it work" for me.
RCS file: /export/00/cvsroot/Zope/lib/python/ZODB/DB.py,v
retrieving revision 1.1.1.11
diff -u -r1.1.1.11 DB.py
--- DB.py 2003/07/24 08:35:40 1.1.1.11
+++ DB.py 2003/07/30 08:57:02
@@ -315,6 +315,8 @@
version=connection._version
# Update modified in version cache
# XXX must make this work with list or dict to backport to 2.6
+if type(oids) is type(''):
+oids = { oids: 1 }
for oid in oids.keys():
h=hash(oid)%131
o=self._miv_cache.get(h, None)
Note that despite the comment, it's in fact getting passed a single
oid as a string, not a dict or a list. I have no idea if there's _also_
code that might be passing a list, which will still be broken.
I'm not committing this to the CVS yet, because I'm not 100% positive
it's the "correct" fix. It certainly _seems_ to be, and I've now got
ZEO clients that actually work worth a damn...
Anthony
___
Zope-Dev maillist - [EMAIL PROTECTED]
http://mail.zope.org/mailman/listinfo/zope-dev
** No cross posts or HTML encoding! **
(Related lists -
http://mail.zope.org/mailman/listinfo/zope-announce
http://mail.zope.org/mailman/listinfo/zope )
